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CHAPTER 1 


Introduction 

1.1. Overview of the Teacher Education in Mathematics Study^ 

The Teacher Education Study in Mathematics or TEDS-M is a study conducted 
under the aegis of the International Association for the Evaluation of Educational 
Achievement (lEA). The lead research center for the study is the International Study 
Center at Michigan State University (ISC/MSU). The ISC/MSU worked from 2006 to 
2011 with the International Study Center at the Australian Council for Educational 
Research (ACER) and the National Research Coordinators (NRCs) in 17 countries to 
produce this study. TEDS-M received funding from lEA, the United States of America 
National Science Eoundation, and the collaborating countries. 

1.1.1. Purpose of TEDS-M 

TEDS-M focused on how teachers are prepared to teach mathematics in primary and 
lower-secondary school, and studied the variation in the nature and impact of teacher 
education programs within and across countries. TEDS-M is the first cross-national 
study to provide data on the knowledge that future primary and lower-secondary school 
teachers have acquired in their mathematics teacher education. The study collected and 
analyzed data from representative samples of pre-service teacher education programs, 
their future primary and lower-secondary teachers, and their teacher educators from 
participating countries to inform policy and practice in teacher education. The key 
research questions for the study focused on the relationships between teacher education 
policies, institutional practices, and future teacher mathematics and pedagogy 
knowledge. 

1.1.2. Research Questions 

What are the policies that support primary and secondary teachers’ achieved level 
and depth of mathematics and related teaching knowledge? 

The study examined the policies directed at mathematics teachers, including 
recruitment, selection, preparation, and certification. Specifically these comprise (a) 
the policies that regulate and influence the design and delivery of mathematics teacher 
education for elementary and lower-secondary teachers; (b) the characteristics of 
institutions and programs charged with implementing these policies; (c) the countries’ 
distinctive political, historical, and cultural contexts as they influence policy and practice 
in mathematics teacher education; and (d) the policies in each country regarding 
standards for degrees, coverage of topics, certification practices, recruitment, selection, 
and preparation of future mathematics teachers. 


1 The first section of this introduction is a shorter version of Chapter 1 found in Policy, Practice, and Readiness 
to Teach Primary and Secondary Mathematics in 17 Countries. Findings from the lEA Teacher Education and 
Development Study in Mathematics (TEDS-M). (Tatto, Schwille, Senk, Ingvarson, Rowley, Peck, Bankov, Rodriguez 
& Reckase, 2012). 
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What learning opportunities, available to prospective primary and lower- secondary 
mathematics teachers, allow them to attain such knowledge? 

The study examined the intended and implemented curriculum of teacher education 
at the institutional level, as well as the overall opportunities to learn embedded in 
this curriculum. Specifically these comprise (a) the kinds of institutional and field- 
based opportunities provided for future primary and lower-secondary teachers; (b) 
the programs’ enacted curriculum and standards; (c) the content taught in teacher 
education programs and the organization of instruction; and (d) the qualifications and 
prior experiences of those responsible for the implementation of these programs. 

What are the level and depth of the mathematics and related teaching knowledge 
attained by prospective primary and lower-secondary teachers at the end of their 
pre-service teacher education? 

The study examined the intended and achieved goals of teacher education. Specifically 
these comprise (a) the content knowledge of mathematics that future teachers have 
acquired as an outcome of teacher education; (b) the depth of understanding that they 
achieved; (c) the knowledge for mathematics teaching (e.g., content, pedagogy, and 
curriculum) that future teachers achieved at the end of their teacher education and 
once they are considered “ready to teach;” (d) other characteristics to help explain their 
ability to master this knowledge; and (e) the beliefs about the nature of mathematics 
and about teaching and learning mathematics at the end of their preparation. 

1.1.3. TEDS-M Design 

Data Sources 

The first research question was addressed through individual case study country reports, 
questionnaires, and interviews issued by the TEDS-M International Study Center. 
The second and third research questions were answered via four surveys of nationally 
representative samples of (1) teacher education institutions and programs; (2) teacher 
educators; (3) future primary teachers who received preparation to teach mathematics; 
and (4) future lower- secondary teachers also preparing to teach mathematics. 

Sample 

In most countries, TEDS-M implemented a two-stage sampling design. First, the 
Sampling Unit of the lEA DPC, working with the national research centers in each 
participating country, selected samples. These were representative of the national 
population of teacher preparation (TP) institutions that offered education to the target 
population of future teachers who were prepared to teach mathematics at the primary 
or secondary level. Once an institution was selected, all programs associated with the 
mathematics preparation of future primary and lower-secondary teachers were included 
in the survey. Then, within these institutions and programs, samples of educators and 
of future teachers were surveyed. In many countries, it was necessary to select all TP 
institutions to reach lEA’s sampling standards. In almost all countries, it was necessary 
to survey all eligible educators for TEDS-M. Likewise, in many countries, all eligible 
future teachers in the sampled institutions were surveyed. All samples were drawn using 
randomization. A software package (WinW3S) provided by the lEA was used in each 
national center to select the samples of future teachers, classes, and educators. Sampling 
errors were computed using balanced half-sample repeated replication (or BRR, a well- 
established re-sampling method). All countries participating in TEDS-M were required 
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to provide complete national coverage of their national desired target populations. 
However, in some cases, organizational or operational conditions made it difficult 
for some NRCs to obtain complete national coverage. The conceptual framework, 
design, and methodology for the study have been thoroughly documented in various 
documents and reports (see Tatto, Schwille, Senk, Ingvarson, Peck, and Rowley, 2008; 
Tatto et al., 2012). 

1.1.4. Participating Countries 

The 17 countries that participated in the TEDS-M study include Botswana, Canada 
(four provinces), Chile, Chinese Taipei, Georgia, Germany, Malaysia, Norway, Oman 
(lower-secondary teacher education), the Philippines, Poland, Russia, Singapore, Spain 
(primary teacher education), Switzerland (German- speaking cantons), Thailand, and 
the USA (public institutions, concurrent and consecutive routes only). 

1.1.5. Target Audiences for TEDS-M 

TEDS-M was designed to inform policy and practice in mathematics teacher education. 
Eor educational policymakers TEDS-M contributes data on institutional arrangements 
that are effective in helping teachers become sufficiently knowledgeable in mathematics 
and related teaching knowledge. Eor teacher educators who design, implement, and 
evaluate teacher education curriculum, TEDS-M contributes a shared language, a 
shared database, and benchmarks for examining teacher education design against what 
has proved possible and desirable to do in other settings. Eor mathematics educators, 
TEDS-M provides a better understanding of what qualified teachers of mathematics are 
able to learn about the content and pedagogy of mathematics, as well as the arrangements 
and conditions needed for acquisition of this knowledge. Eor educators in general and 
for informed laypersons, TEDS-M provides a better understanding of how and what 
teachers learn as they prepare to teach. 

1.2. Overview of the TEDS-M International Database and the User 
Guide for the International Database 

To support and promote secondary analyses, the lEA is making the TEDS-M 
international database, the TEDS-M Technical Report (Tatto, 2012) and the TEDS-M 
User Guide available to analysts and public users.^ The database comprises future teacher 
preparation program-, educator-, and future teacher level data from 17 countries. 

This guide has been designed to provide a basic yet thorough introduction to the 
TEDS-M international database and to the most commonly used descriptive and 
inferential analyses. The guide, however, was not designed to include and illustrate 
every possible analytical technique appropriate for TEDS-M and therefore does not 
describe, for example, the multi-level modeling of data. The guide is further limited 
to the use of SPSS (IBM Inc., 2011) which is a widely used package for the analysis 
of large-scale educational datasets. Still, users wishing to undertake advanced analysis 
not covered by this guide or those wishing to use another statistical software packages 
such as SAS, WesVar, or Stata will find sufficient information on the database and its 
technical aspects to successfully configure both software and statistical models. 


2 Policy, Practice, and Readiness to Teach Primary and Secondary Mathematics in 1 7 Countries. Findings from the lEA 
Teacher Education and Development Study in Mathematics (TEDS-M). (Tatto et al., 2012) is available on the lEA 
website and includes a detailed description of the study, its findings, and methods. 


10 


TEDS-M IDE USER GUIDE 


1.2.1. Country Acronyms and Identification Numbers 

Operational country acronyms and identification numbers were based on the ISO 
3166-1 definition for country codes. A three-letter operational alphanumeric code is 
used in filenames and identifies the country associated with that file. The operational 
numeric code IDCNTRY is used within the data files to identify the country for each 
record. For countries or sub-national entities not listed in the ISO 3166-1 standard the 
lEA Data Processing and Research Center (DPC) derived new operational alphanumeric 
and associated numeric codes. In TEDS-M this applies to Norway as the three program 
types there (“ALU”, “ALU-t”, and “PPU & Masters”) are partly overlapping populations 
and hence had to be identified individually. Analysis across these program types is 
inappropriate due to this overlap (see Appendbc B of the TEDS-M International Report 
[Tatto et al., 2012]). To avoid the risk of inappropriate analysis, data from Norway is 
treated as three different entities with three different numeric country identification 
codes (see Table 1.1 below). 

With the exceptions of Oman, where there were no future primary teachers enrolled in 
teacher preparation programs during the time of the TEDS-M study, and Spain, which 
opted to survey only programs for future primary teachers, all countries collected and 
submitted data for the primary and lower-secondary teacher education levels. 

Table 1.1 below lists the country names and operational codes of the TEDS-M 
participants. 


Table 1.1: Country Names and Operational Codes of the TEDS-M Participants 


Alpha-3 Code 

Numerical Code 

Country Name 

BWA 

72 

Botswana 

CAN 

124 

Canada 

CHL 

152 

Chile 

TWN 

158 

Chinese Taipei 

GEO 

268 

Georgia 

DEU 

276 

Germany 

MYS 

458 

Malaysia 

OMN 

512 

Oman 

PHL 

608 

Philippines 

POL 

616 

Poland 

RUS 

643 

Russian Federation 

SGP 

702 

Singapore 

ESP 

724 

Spain 

CHE 

756 

Switzerland 

THA 

764 

Thailand 

USA 

840 

United States of America 

NOR 

57891 

Norway (ALU) 

NOR 

57892 

Norway (ALU+) 

NOR 

57893 

Norway (PPU & Masters) 
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1.2.2. Limitations of the Public-use Database 

There are some limitations to the use of the data that need to he carefully considered 

when analyzing the TEDS-M data. Researchers are strongly advised to familiarize 

themselves with the study design and the specifics of the national samples (see also the 

TEDS-M Technical Report [Tatto, 2012] for details). 

General 

• Canada: Data were unacceptable due to low participation rates. Data are included in 
the database with weights set to 0. Inferences from sample data to populations are 
inappropriate. 

• Norway: Program types are reported separately (see 1.2.1) because the populations 
partly overlap; data from these program types cannot be aggregated. 

• Oman: Only secondary teacher education was provided at the time of testing. 

• Russian Federation: Secondary pedagogical institutions were not covered. 

• Spain: Only primary teacher education was covered. 

• Switzerland: Only institutions where German is the primary language of use and 
instruction were covered. 

• USA: Only public institutions were covered. 

Institutional Program Data 

• Canada: Data were unacceptable. 

• Chinese Taipei: Exclusion rate is greater than 5 percent (very small institutions were 
excluded). 

• Germany: Did not authorize reporting the institutional program questionnaire 
data. 

• Malaysia: Participation rate is 57 percent; quality of the institutional program 
questionnaire data is questionable.^ 

• Norway: Program types are reported separately (see 1.2.1) because the populations 
partly overlap; data from these program types cannot be aggregated. 

• Oman: Only secondary teacher education was provided at the time of testing. 

• Philippines: Exclusion rate is greater than 5 percent (very small institutions were 
excluded). 

• Poland: Institutions with consecutive programs only were not covered. 

• Russian Federation: Secondary pedagogical institutions were not covered. 

• Spain: Only primary teacher education was covered. 

• Switzerland: Only institutions where German is the primary language of use and 
instruction were covered. 

• USA: Only public institutions were covered. 

Educator Data 

• Canada: Data were unacceptable. 

• Chile: Combined participation rate is 54 percent. 

• Germany: Combined participation rate is 56 percent; survey of institutions and 
future teachers are not connected with survey of educators. 

• Malaysia: Combined participation rate was 57 percent. 


3 lEA standards consider <60 percent as low participation rates. 
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• Norway: Data were unacceptable. 

• Poland: Combined participation rate was between 60 and 75 percent; institutions 
with consecutive programs only were not covered. 

• Russian Federation: Secondary pedagogical institutions were not covered. 

• Spain: Only primary teacher education was covered. 

• Switzerland: Combined participation rate was 52 percent; only institutions where 
German is the primary language of use and instruction were covered. 

• United States: Data were unacceptable. 

Future Primary Teacher Data 

• Botswana: The sample size is small (N=86), but comes from a census of a small 
population. 

• Canada: Data were unacceptable. 

• Chile: Combined participation rate is between 60 and 75 percent. 

• Germany: Due to data protection restrictions in Germany institution and program 
identification variables in the future teacher data files have been made unusable so 
that future teacher data cannot be grouped by teacher preparation institutions or 
programs. A list matching future teachers to their institutions and programs can he 
obtained on demand from the TEDS-M National Study Center (tedsm@staffhu- 
berlin.de). 

• Norway: Combined participation rate is between 60 and 7 5 percent. An exception was 
made to accept data from one institution because one additional participant would 
have brought the response rate above the 50 percent threshold. Program types “ALU” 
and “ALU-t” are reported separately because the two populations partly overlap; data 
from these program types cannot be aggregated. 

• Poland: Combined participation rate is between 60 and 75 percent; institutions with 
consecutive programs only were not covered. 

• Russian Federation: Secondary pedagogical institutions were not covered. 

• Switzerland: Only institutions where German is the primary language of use and 
instruction were covered. 

• USA: Only public institutions were covered. Combined participation rate is between 
60 and 75 percent. An exception was made to accept data from two institutions 
because, in each case, one additional participant would have brought the response 
rate above the 50 percent threshold. Although the participation rate for the complete 
sample meets the required standard, the data contain records that were completed 
using a telephone interview, when circumstances did not allow administration of 
the full questionnaire. Of the 1501 recorded as participants, the full questionnaire 
was administered to 1 185. Bias may arise in the data because significant numbers of 
individuals were not administered the full questionnaire. 

Future Lower- Secondary Teacher Data 

• Botswana: The sample size is small (N=53), but comes from a census of a small 
population. 

• Chile: Combined participation rate between 60 and 75 percent. 

• Georgia: Combined participation rate between 60 and 75 percent; an exception 
was made to accept data from two institutions because, in each case, one additional 
participant would have brought the response rate above the 50 percent threshold. 
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• Germany: Due to data protection restrictions in Germany institution and program 
identification variables in the future teacher data files have been made unusable so 
that future teacher data cannot be grouped by teacher preparation institutions or 
programs. A list matching future teachers to their institutions and programs can be 
obtained on demand from the TEDS-M National Study Center (tedsm@staff.hu- 
berlin.de). 

• Norway: Combined participation rate is 58 percent. Program types “ALU”, “ALU-t”, 
and “PPU & Masters” are reported separately because the populations partly overlap; 
data from these program types cannot be aggregated. 

• Poland: Combined participation rate is between 60 and 75 percent; institutions with 
consecutive programs only were not covered. 

• Russian Federation: An unknown number of those surveyed had previously qualified 
to become primary teachers. 

• Switzerland: Only institutions where German is the primary language of use and 
instruction were covered. 

• USA: Only public institutions were covered. Combined participation rate is between 
60 and 7 5 percent. An exception was made to accept data from one institution because 
one additional participant would have brought the response rate above the 50 percent 
threshold. Although the participation rate for the complete sample meets the required 
standards, the data contain records that were completed using a telephone interview, 
when circumstances did not allow administration of the full questionnaire. Of the 
607 recorded as participants, the full questionnaire was administered to 502. Bias may 
arise in the data because significant numbers of individuals were not administered 
the full questionnaire. 

1.2.3. TEDS-M Instruments 

The study administered four questionnaires to respondents in institutions: an 
institutional program questionnaire, an educator questionnaire, and two different 
versions of the future teacher questionnaire: one for future primary level teachers and 
one for future lower-secondary level teachers. The latter two included achievement 
items as well as background questions. 

All questionnaires were translated into the respective language(s) and verified for 
linguistic equivalence. To conduct a valid analysis of the TEDS-M data and to correctly 
interpret the results, it is important to be aware of adaptations to the questionnaires 
that might have been made by national centers. In some instances, tbe international 
version of a question or item had to be adapted by all participating countries, for 
example with respect to ISCED levels. In other cases, national centers adapted questions 
or items for socio-cultural reasons or because the international version was partly or 
completely not applicable in their country. All the adaptations were refereed by the 
International Study Center at Michigan State University to secure as much integrity and 
consistency as possible across questionnaires and countries. Users of the international 
database are encouraged to refer to Supplement 2 of this guide for more details about 
national adaptations in TEDS-M and potential consequences for comparability across 
countries. 
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1.3. Analyzing the TEDS-M Data 

TEDS-M uses item response theory (IRT) scaling to summarize future teacher 
achievement on the assessment. Proficiency scores in mathematics and mathematics 
pedagogy content knowledge have been estimated separately for future primary and 
for future lower- secondary teachers. Furthermore, each future teacher record in the 
TEDS-M international database contains several scale scores for the opportunities 
to learn (OTL) and future teacher beliefs scales (see Appendix B of the TEDS-M 
International Report [Tatto et. al., 2012] for details). 

lEA developed the International Database (IDB) Analyzer software (lEA, 2011) 
specifically for analyzing TEDS-M international data files. This software is designed to 
support users in the analysis of the TEDS-M data taking into account the sample design 
and providing estimates of the precision of the results. The lEA IDB Analyzer applies 
the balanced repeated replication (BRR) method to provide appropriate standard errors 
for each statistic. It also simplifies management of the TEDS-M database by providing 
a module for selecting subsets of countries and variables, and merging files for analysis. 
Chapter 4 of this user guide provides detailed instruction on how to analyze the TEDS-M 
data using the lEA IDB Analyzer. 

1.3.1. Unit of Analysis (Program Grouping) 

When analyzing TEDS-M data, users need to carefully take into account the differences 
in the teacher preparation programs within and across countries, with respect to 
content and outcome of the programs. Therefore, programs have been grouped to 
allow for comparison across countries (see Chapter 1.5 of Appendix B of the TEDS-M 
International Report [Tatto et. al., 2012] for further details). In TEDS-M analysis across 
countries should therefore always be at this program group level. 

The variables TARGET? (for the future primary teacher data) and TARGETS (for the 
future lower-secondary teacher data) contain information about the grouping of the 
different teacher education programs for future teachers of mathematics within and 
across countries. Either one of these two variables should always be used as a grouping 
variable when comparing data across countries in TEDS-M. 

Table 1.2 lists these program groups used in TEDS-M as the unit of analysis. 


Table 1.2: TEDS-M Program Groups 


Level 

Program 
Group Variable 

Program Group 
Variable Code 

Program Group Name 

Primary 

TARGETP 

1 

Lower-Primary Generalists 
(Grade 4 maximum) 

2 

Primary Generalists (Grade 6 maximum) 

3 

Primary/Lower-Secondary Generalists 
(Grade 10 maximum) 

4 

Primary Mathematics Specialists 

Lower 

Secondary 

TARGETS 

5 

Lower-Secondary (Grade 10 maximum) 

6 

Lower & Upper Secondary 
(to Grade 11 and above) 
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1 .4. Contents of the TEDS-M User Guide for the International 
Database 

The TEDS-M User Guide describes the content and format of the TEDS-M international 

database. In addition to this introduction the TEDS-M User Guide includes the 

following chapters: 

• Chapter 2 describes the structure and the content of the TEDS-M international 
database. 

• Chapter 3 provides information about the sampling weights and variance estimation 
procedures. 

• Chapter 4 introduces the lEA International Database (IDE) Analyzer software (lEA, 
2011) and presents examples of analyses of TEDS-M data using this software in 
conjunction with SPSS. 

The TEDS-M User Guide is accompanied by the following supplements: 

• Supplement 1 comprises the international version of the TEDS-M questionnaires. 

• Supplement 2 describes any adaptations to the questions in the questionnaires made 
by the TEDS-M participants. 

• Supplement 3 lists all indices and other derived variables created for reporting the 
TEDS-M questionnaire data. 

• Supplement 4 includes those items of the TEDS-M primary and lower-secondary 
mathematics and mathematics pedagogy knowledge assessment that have been 
released to the public. 


1.5. Contents of the TEDS-M International Database Package 

In addition to the data files from the TEDS-M survey from participating countries, the 
TEDS-M international database contains supplementary material and documentation 
supporting users of the international database and providing necessary and 
complementary information for analysis: 

Data: Institutional program, educator, and future teacher data in SPSS and 

SAS format 

User Guide: This user guide with supplements 

Codebooks: Codebook files with descriptive information on all variables in the 

international database files 


Almanacs: 


Programs: 


Data almanacs with summary statistics for the future teacher knowledge 
items and contextual questionnaire variables 

SPSS syntax programs and macros to support analysis 


The TEDS-M international database and the lEA IDE Analyzer are available at the lEA 
study data repository at: 
http://www.iea.nl/ data.html 
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CHAPTER 2 

The TEDS-M International Database Files 


2.1. Overview 

The TEDS-M international database contains future teacher knowledge scores and 
future teacher, educator, and institutional program contextual data collected in 17 
countries. This chapter describes the contents of the database. 

The data are made available as separate files for each country and each population 
(future primary and lower-secondary teachers, and future teacher educators) or level 
(programs within institutions). 

Measures were implemented for the public-use international database to protect the 
identity of respondents and to maintain the confidentiality of the information for 
all participating countries (see Chapter 10 of the TEDS-M Technical Report [Tatto, 
2012]). Individual country procedures were implemented in compliance with internal 
regulations. 

2.2. TEDS-M Main Data Files 

The TEDS-M database includes the actual data from all questionnaires administered 
to the institutions, educators, and future teachers. These data files also include the 
knowledge scores estimated for participating future teachers, as well as background 
variables derived for reporting in the TEDS-M international reports. 

This section describes the contents and format of the TEDS-M data files. The data 
files are provided in SPSS format (.SAV) and SAS export format (.EXP). Data files are 
provided for each country that participated in TEDS-M. The file names given to the 
various data file types are shown in Table 2.1. Eor each file type, a separate data file is 
provided for each participating country. Eor example, DlGCHLTl.SAV is an SPSS file 
that contains Chile’s TEDS-M institutional program questionnaire data. All data files 
and the variables they contain are described in the following sections. 

The SPSS files include full dictionary and meta information, i.e. variable name, format 
(type, width, and decimals), label, value labels, missing values, and the measurement 
level set (nominal, ordinal, or scale). The dictionary information can be accessed as a 
list through the SPSS View Variables menu or in SPSS output format selecting the 

File Display Data File Information menu. 


18 


TEDS-M IDE USER GUIDE 


Table 2. 1 TEDS-M Data File Names 


File Names 

Description 

DIG***T1 

Institutional Program Data file 

DEG* • «T1 

Educator Questionnaire Data file 

DPG* • «T1 

Future Primary Teacher Questionnaire Data file 

DSG* • «T1 

Future Lower-Secondary Teacher Questionnaire Data file 

DPR* * *T1 

Reliability Scoring Future Primary Teacher Questionnaire Data file 

DSR* * *T1 

Reliability Scoring Future Lower-Secondary Teacher Questionnaire Data file 


••• = 3-character alphanumeric country code based on the ISO 3166 coding scheme (see Table 1.1) 


2.2.1. TEDS-M Future Teacher Data Files 

The TEDS-M future teacher files consist of both responses to background questions 
and individual achievement items. Each future teacher questionnaire consists of four 
different parts. Parts A, B, and D are the same for all future teachers and only Part C - 
containing the assessment items - is different for future primary and lower-secondary 
teachers. 

Part A: General background 

Part B: Opportunity to learn 

Part C: Mathematics and mathematics pedagogy content knowledge for teaching; 

consisting of either items for future primary or lower-secondary teachers 

Part D: Beliefs about mathematics and teaching 

future primary teachers who participated in TEDS-M were administered one of five 
assessment booklets, and future lower- secondary teachers answered one of three 
assessment booklets. Assessment items were multiple-choice, complex multiple- 
choice, and constructed-response items. The future teacher data files contain the actual 
responses to the multiple- choice and complex multiple-choice items and the scores 
assigned to the constructed-response items. 

2.2.2. TEDS-M Teacher Educator Data Files 

The educators with regular, repeated responsibility for teaching the future teachers who 
were sampled for TEDS-M were administered a questionnaire with questions about 
their background and their teaching practices in the courses they taught to future 
teachers. 

Note that the educator data file cannot and should not be merged with the future teacher 
data files because there is no direct link between them and samples for both populations 
were drawn independently. Some of the surveyed educators might never have taught or 
seen the future teachers who participated in TEDS-M. Merging institutional program 
and educator data is also not possible. In most countries, some educators teach in 
more than one program or even institution (see the TEDS-M Technical Report [Tatto, 
2012]). For reference, the variable TPUIDS in the educator data files lists the programs 
educators teach in with their program identification number (IDTPU). 

Chapter 4 of this user guide describes some analyses with educator data using the lEA 
IDB Analyzer software. 
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2.2.3. TEDS-M Institutional Program Data Files 

The institutional program data files contain program specific responses from institutions 
to the questions in the TEDS-M institutional program questionnaire. Although 
institution level analyses with the institutions as the units of analysis can he performed, 
it is preferable to analyze institution-level variables as attributes of future teachers. To 
perform future teacher level analyses with institutional data, the institutional program 
data files must be disaggregated to future teacher level and merged with the future 
teacher data files using the country and institution identification variables. Details of 
the merging procedure using the TEA IDB Analyzer are described in Chapter 4 of this 
user guide. 

2.2.4. Knowledge Assessment Items 

Survey Items in Part C of the Future Teacher Booklets 

The assessment items have been assigned a special nomenclature for identification (e.g., 
MFC604A1). 

• The first character represents the study phase. It is “M” for Main Study for all 
international variables. 

• The second character, “F”, indicates that the item is located in the future teacher 
questionnaire. 

• The third character indicates the part of the questionnaire where the item is 
located: 

- C: Part C 

• The fourth character indicates the number of the booklet where the item is located: 

- 1: Booklet PMl (Future Primary Teacher Booklet 1) 

- 2: Booklet PM2 (Future Primary Teacher Booklet 2) 

- 3: Booklet PM3 (Future Primary Teacher Booklet 3) 

- 4: Booklet PM4 (Future Primary Teacher Booklet 4) 

- 5: Booklet PM5 (Future Primary Teacher Booklet 5) 

- 6: Booklet SMI (Future Lower-Secondary Teacher Booklet 1) 

- 7: Booklet SM2 (Future Lower-Secondary Teacher Booklet 2) 

- 8: Booklet SM3 (Future Lower-Secondary Teacher Booklet 3) 

• The fifth and sixth characters indicate the number of the item within the booklet. 

• The seventh character represents the sub-item within the item number (if 
applicable). 

• The eighth character indicates the sub-part of the sub-item (if applicable). 

Item Response Code Values 

A series of conventions were adopted to code the data included in the TEDS-M data files. 
This section describes these conventions for the knowledge items. The values assigned 
to each of the knowledge item variables also depend on the item format. For multiple- 
choice items, numerical values from 1 through 5 are used to correspond to the response 
options A through E, respectively. For these items, the correct response is included in 
the codebook file (“MEAS_CLASS” in ’*^.sdb file and “CAT” in ’*^.pdf file). 

As a result of psychometric analysis, some derived variables have been added to the 
future teacher data files that combine two or three items and for which a score has been 
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assigned depending on the response of the items to be combined. If the single items 
were multiple-choice, the codes for the derived variables were 2 if all single items were 
answered correctly (“correct response”) and 1 if at least one of the single items was 
answered correctly (“partially correct response”). 

Each constructed-response item had its own scoring guide that relied on a two-digit 
scoring scheme to provide diagnostic information. The first digit designated the 
correctness level of the response: 2 for a two-point response, 1 for a one-point response, 
and 7 for an incorrect response. The second digit, combined with the first, represented 
a diagnostic code used to identify specific types of approaches, strategies, or common 
errors and misconceptions in responding to the item. A second digit of 0 through 5 
was used for pre-defined international codes at each correctness level, while a second 
digit of 9 corresponded to “other” types of responses that fell within the appropriate 
correctness level, but did not fit any of the pre-defined international codes. A special 
code, 99, was used for completely blank responses. 

Code for Missing Values 
Omitted or Invalid Response 

The response to a question is coded as omitted when the question was administered but 
not answered or when an invalid response was given. The following cases are coded as 
an omitted or invalid response: 

• Question/item non-response: The question was administered but no response was 
provided. 

The code for “omitted/invalid” responses in SPSS files is “9” or “99” (depending on the 
field length of the variable). The code for “omitted/invalid” responses in SAS files is “.”. 

Not Administered 

A response to a question is coded as “not administered” when: 

• Unit non-response: All variables in not returned or empty questionnaire are coded 
as “not administered.” 

• Questions or parts not administered: A country may have chosen not to administer 
a certain question in their national questionnaire as documented in the national 
adaptation form (NAF) that lists all agreed-upon deviations from the international 
version of the questionnaire. The variable(s) corresponding to the left out question 
are coded as “not administered.” 

• Item left out or misprinted: If a particular question or item (or maybe a whole 
page) was misprinted or left out of a questionnaire, or for other reasons was not 
available for the respondent to answer, the corresponding variable(s) are coded as 
“not administered.” 

The code for “not administered” questions in SPSS files is “SYSMIS” (.). The code for 
“not administered” questions in the SAS files is “.A”. 

Not Reached (Part C of Future Teacher Booklets) 

A special missing code was assigned to questions that were deemed “not reached” to 
further distinguish them from “omitted” responses during item calibration and scoring. 
“Omitted” questions are those that a respondent most likely read, but either consciously 
decided not to answer or accidentally skipped; that is, the respondent started answering 
the questions but stopped answering before the end of the questionnaire, most likely 
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due to a lack of time. “Not reached” variables are exclusively located towards the end of 
questionnaire. “Not reached” questions can be seen as a variation or sub-class of omitted 
questions. To code an item as “not reached” in TEDS-M, the following algorithm was 
used: 

• The last valid answer provided by the respondent in Part C was identified. 

• The first omitted response after this last response was coded as “omitted”. 

• All following responses until the end of Part C were then coded as “not reached.” 

- For example, in SPSS files the response pattern “1 9 1 2 9 9 9 9 9 9” (where “9” 
represents “omitted”) is recoded to “1 9 1 2 9 6 6 6 6 6” (where “6” represents “not 
reached”). 

- When recoding “omitted” values to “not reached,” all “not administered” values 
were maintained. For example the pattern “3 1 42999.99” would be recoded to 
“3 1 42966.66.” 

In SPSS files, “not reached” variables are coded as “6”, “96”, “996”, and so on (depending 
on the field length of the variable). In SAS data files the code for “not reached” is “.R”. 

TEDS-M Knowledge of Mathematics and Mathematics Pedagogy Scores 

In TEDS-M, future teachers’ knowledge of mathematics and mathematics pedagogy was 
assessed. Item Response Theory (IRT) was used to estimate the mathematics content 
knowledge (variable MCK) and mathematics pedagogy content knowledge (variable 
MPCK) of future teachers. The MCK and MPCK scale scores were calculated on the 
logit scale and then standardized to have a mean of 500 and a standard deviation of 
100 for equally weighted countries (see Appendix B of the TEDS-M Technical Report 
(Tatto, 2012) for more information on the scaling procedures). 

NOTE: Although standardized to the same mean and standard deviation, the MCK 
scores for future primary teachers are not comparable to the MCK scores for future 
lower-secondary teachers as they answered different items according to whether they 
were expected to become qualified to teach at the primary or lower-secondary level. 
Thus scaling was done separately for both populations. The same applies to the MPCK 
scores. 

TEDS-M International Benchmarks of Knowledge (Anchor Points) 

To help readers interpret scores on these scales, TEDS-M researchers identified key 
points on the scales, called anchor points. The anchor points do not represent a priori 
judgments about whether a given scale score is good or bad. Rather, they are descriptions 
of the performance of future teachers with scores at specific points on the scale. Items 
at the anchor points were determined by the probability that a person with that score 
would get the item right. Future teachers with scores at the anchor point 1 are able to 
provide correct answers to items classified at that point or below with a probability of 
0.70 or greater. Hence, sets of such items were used to develop descriptions of what 
future teachers at (or above) the anchor points were likely to achieve. Items in anchor 
point 2 were items that future teachers were likely to answer correctly with a probability 
of less than 0.50. A panel of mathematicians and mathematics educators analyzed the 
items classified at these anchor points and formulated descriptions of the knowledge of 
future teachers at each point. 
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Two anchor points were defined for the primary level MCK scale. Anchor point 1 
represents a lower level of MCK and corresponds to a scale score of 431. Anchor point 
2 represents a higher level of knowledge corresponding to a scale score of 516. Because 
of the relatively small number of items measuring mathematics pedagogy content 
knowledge, only one anchor point was defined at the primary level MPCK scale, ft 
represents a score of 544 on the MPCK scale. 

Similarly, two anchor points were selected for the lower-secondary MCK scale. Anchor 
point 1 represents a lower level of performance and corresponds to a scale score of 490. 
Anchor point 2 represents a higher level and corresponds to a scale score of 559. As was 
true for the primary level, because of the relatively small number of items measuring 
mathematics pedagogy content knowledge, only one anchor point for MPCK was 
defined at the lower-secondary level, ft corresponds to a scale score of 509. 

For more details about the development and meaning of the anchor points see Chapter 
5 and Appendix B of the TEDS-M International Report (Tatto et ah, 2012) and the 
TEDS-M Technical Report (Tatto, 2012). 

2.2.5. Questionnaire Variables 
Survey Questionnaire Naming Convention 

All variables created to enter the responses in the survey instruments follow naming 
conventions tailored to the type of survey instrument from which they originate. 

Institutional Program 

The variables (or questions) in the institutional program questionnaire have been 
assigned a special nomenclature for identification (e.g., M1B003A2). 

• The first character represents the study phase; it is “M” for Main Study for all 
international variables. 

• The second character indicates the type of questionnaire. “1” stands for the 
institutional program questionnaire. 

• The third character indicates the part of the questionnaire. There are Parts A, B, C, 
D, E, F, G, Fi, and 1. 

- A: Part A: Program Description 

- B: Part B: Future Teacher Background 

- C: Part C: Selection Policies 

- D: Part D: Program Content 

- E: Part E: Field Experience 

- F: Part F: Program Accountability and Standards 

- G: Part G: Staffing 

- Fi: Part Fi: Program Resources 

- I: Part I: Reflections on the Program 

• The fourth to eighth characters indicate the number of the question in the 
questionnaire. 
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Educators 

The variables (or questions) in the educator questionnaire have been assigned a special 
nomenclature for identification (e.g., MEA003E). 

• The first character represents the study phase; it is “M” for Main Study for all 
international variables. 

• The second character indicates the respondent. The letter “E” is used for educators. 

• The third character indicates the part of the questionnaire: 

- A: Part A: General and Academic Background 

- B: Part B: Teaching Background 

- C: Part C: Professional Experience 

- D: Part D: Research Experience 

- E: Part E: Eield-Based Instruction 

- E: Part E: Opportunities to Learn in Your <Course> 

- G: Part G: Opportunities to Learn (Mathematics and Mathematics <Pedagogy> 
Educators 

- H: Part H: Opportunities to Learn (General <Pedagogy> Educators) 

- I: Part 1: Opportunities to Learn (All Educators) 

- J: Part J: Coherence of the Teacher Preparation Program (All Educators) 

- K: Part K: Beliefs about Mathematics (All Educators) 

- L: Part L: Preparedness for Teaching Mathematics (All Educators) 

• The fourth to eighth characters indicate the number of the question within the 
questionnaire. 

Future Teachers - Context Questions (Part A, B, and D) 

The variables (or questions) in the future teacher questionnaire were assigned a special 
nomenclature for identification (e.g., MEA004D). 

• The first character represents the study phase, ft is “M” for Main Study for all 
international variables. 

• The second character indicates the respondent. The letter “E” is used for future 
teachers. 

• The third character indicates the part of the questionnaire: 

- A: Part A: General Background 

- B: Part B: Opportunities to Learn 

- D: Part D: Beliefs about the Nature of Mathematics 

• The fourth to eighth characters indicate the number of the question within the 
questionnaire. 

Survey Questions Response Code Values 

There are three different types of questions in the TEDS-M survey questionnaires: 

• Categorical questions: Numerical values which corresponded to the response 
options of the question were entered (1st option -1, 2nd option -2,...). 

• Non categorical questions: The exact values given by the respondents were entered. 

• Open ended text responses: These responses are not part of the international 
database and were only used for national purposes. 
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Codes for Missing Values 
Omitted or Invalid Response 

The response to a question is coded as omitted when the question was administered hut 
not answered or when an invalid response was given. The following cases are coded as 
an omitted or invalid response: 

• Question/item non-response: The question was administered but no response was 
provided. 

• Uninterpretable/ invalid: The respondent selected more than the expected number of 
checkboxes or gave an uninterpretable or otherwise invalid response to a question. 

The code for “omitted/invalid” responses in SPSS files is “9”, “99”, “999”, and so on 
(depending on the field length of the variable). The code for “omitted/ invalid” responses 
in SAS files is 

Not Administered 

A response to a question is coded as “not administered” when: 

• Unit non-response: All variables in not returned or empty questionnaire are coded 
as “not administered.” 

• Questions or parts not administered: A country may have chosen not to administer 
a certain question in their national questionnaire as documented in the NAF. The 
variable(s) corresponding to the left out question is coded as “not administered.” 

• Item left out or misprinted: If a particular question or item (or maybe a whole 
page) was misprinted or left out of a questionnaire, or for other reasons was not 
available for the respondent to answer, the corresponding variable(s) is coded as “not 
administered.” 

The code for “not administered” questions in SPSS files is “SYSMIS” (.). The code for 
“not administered” questions in the SAS files is “.A”. 

Logically Not Applicable (Institutional Program and Educator Questionnaires, Parts 
A, B, and D of Future Teacher Booklets) 

The response to a variable is coded as “logically not applicable” if: 

• The previous filter question has been answered in a way that makes a response to the 
dependent questions logically impossible; in other words the respondent skipped the 
dependent questions. 

“Logically not applicable” responses are coded in the SPSS files as “6”, “96”, “996”, and so 
on (depending on the field length of the variable). The code for logically not applicable 
responses in SAS files is “.B”. 

2.2.6. Summary Indices and Derived Variables^ 

• MEG2APRA - Mathematics Education Pedagogy - Assessment Practice 

• MEG2IPRA - Mathematics Education Pedagogy - Instructional Practice 

• MEHIIMPR - Teaching For Improving Practice 

• MEH2DVRS - Teaching For Diversity 

• MEH2REFL - Teaching For Reflecting On Practice 

• MEIIPART - Mathematics Education Pedagogy - Class Participation 


1 Details on the calculation and composition of the scales are provided in Supplement 3 of this TEDS-M IDB User 
Guide. For information about the scaling procedures please refer to the TEDS-M Technical Report (Tatto, 2012). 
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• MEIIREAD - Mathematics Education Pedagogy - Class Reading 

• MEI2CLP - School Experience - Connecting Classroom Learning to Practice 

• MEI3AUSE - Mathematics Education Pedagogy - Assessment Uses 

• MEI3IPLA - Mathematics Education Pedagogy - Instructional Planning 

• MEI5SOLV - Mathematics Education Pedagogy - Solving Problems 

• MEJICOH - Program Coherence 

• MEKIRULE - Nature Of Mathematics As Rules And Procedures 

• MEKIPROC - Nature Of Mathematics As Process Of Inquiry 

• MEK2TEAC - Learning Mathematics Through Teacher Direction 

• MEK2ACTV - Learning Mathematics Through Active Learning 

• MEK3EIXD - Mathematic Achievement As Fixed Ability 

• MELIPREP - Preparedness For Teaching Mathematics 

• MEBIGEOM - Tertiary Level Mathematics - Geometry 

• MEBIDISC - Tertiary Level Mathematics - Discrete Structures & Logic 

• MEBICONT - Tertiary Level Mathematics - Continuity & Functions 

• MEBIPRST - Tertiary Level Mathematics - Probability & Statistics 

• MEB2SLMN - School Level Mathematics - Numbers & Measurement & Geometry 

• MEB2SLME - School Level Mathematics - Functions & Probability & Calculus 

• MFB4FOUN - Mathematics Education Pedagogy - Foundations 

• MFB4INST - Mathematics Education Pedagogy - Instruction 

• MFB5PART - Mathematics Education Pedagogy Class Participation 

• MFB5READ - Mathematics Education Pedagogy - Class Reading 

• MFB5SOLV - Mathematics Education Pedagogy - Solving Problems 

• MFB6IPRA - Mathematics Education Pedagogy - Instructional Practice 

• MFB6IPLA - Mathematics Education Pedagogy - Instructional Planning 

• MFB6AUSE - Mathematics Education Pedagogy - Assessment Uses 

• MFB6APRA - Mathematics Education Pedagogy - Assessment Practice 

• MFB7EPSS - Education Pedagogy - Social Science 

• MFB7EPAP - Education Pedagogy - Application 

• MFB8DVRS - Teaching For Diversity 

• MFB8REFL - Teaching For Reflecting On Practice 

• MFB9IMPR - Teaching For Improving Practice 

• MFB13CLP - School Experience - Connecting Classroom Learning to Practice 

• MFB14STR - Supervising Teacher Reinforcement of University Goals for Practicum 

• MFB14STF - Supervising Teacher Feedback Quality 

• MFB15COH - Program Coherence 

• MFDIPROC - Nature Of Mathematics As Process Of Inquiry 

• MFDIRULE - Nature Of Mathematics As Rules And Procedures 

• MFD2TEAC - Learning Mathematics Through Teacher Direction 

• MFD2ACTV - Learning Mathematics Through Active Learning 

• MFD3FIXD - Mathematic Achievement As Fixed Ability 

• MFD4PREP - Preparedness For Teaching Mathematics 

• MFD5QUAL - Quality Of Instruction 

• MCK - Mathematics Content Knowledge 

• MPCK - Mathematics Pedagogy Content Knowledge 
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2.2.7. Structure and Design Variables in TEDS-M Data Files 

Identification Variables 

Due to confidentiality reasons the ID variables in the TEDS-M International Database 

are not connected to the IDs used during data collection preventing the re-identification 

of individuals. 

• IDCNTRY - This numerical variable indicates the ID code for the respective country 
based on the ISO-3166 country code standard. Labels for IDCNTRY are available 
in the SPSS export files based on alphanumeric three-letter acronyms and official 
country names. 

• IDINST - This is the numerical unique ID of the institution that the teacher 
preparation unit is linked to. 

• IDTPU - This is a 4- digit numerical variable that uniquely identifies each participating 
teacher preparation unit. 

• IDEDU - The educator ID identifies individual educators. 

• IDFUT - The future teacher ID uniquely identifies each future teacher in the sample. 
The future teacher ID is used in all the future teacher data files. 

• IDSCORA/IDSCORR - The scorer ID variables are used to uniquely identify 
personnel who scored the constructed-response open-ended items. While IDSCORR 
available in the scoring reliability files identifies the scorer for the reliability scoring, 
IDSCORA available in the future teacher data files identifies the scorer for the main 
scoring. Using these variables permits the identification of systematic errors made by 
individual scorers. 

• ITEDUGRP - Group to which the educator belongs 

- Codel: Mathematics and mathematics pedagogy educator 

- Code 2: Educator who has both responsibilities (mathematics and mathematics 

pedagogy as well as general pedagogy) 

- Code 3: Pedagogy educator 

• TPUIDS - This character variable indicates all TPU IDs that a future teacher educator 
is teaching in and corresponds to the IDTPU values in the DIG file. Multiple values 
are comma- separated, e.g. “1011,1012.” 

• IDBOOK - Each future teacher is administered one of five booklets for future 
primary teachers or one of three booklets for future lower-secondary teachers. This 
variable identifies the booklet that was administered to each future teacher during 
the survey session. 

Tracking Variables 

• ITPART - Einal participation indicator 

• TARGETP - Level at which primary graduates qualify to teach. 

• TARGETS - Level at which lower- secondary graduates qualify to teach. 

• TPULEVEL - Level of future teacher training. 

• ITLANG - This variable indicates in which language the questionnaire was 
administered. This variable has been coded to “ 1 ” for all countries having administered 
the survey in one administration language only. 
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• ILRELIAB - This binary variable indicates whether the future teacher booklet was 
used for reliability scoring. 

- Code 0: Booklet not used for reliability scoring 

- Codel: Booklet used for reliability scoring 

Weights (refer to Chapter 3) 

• INSRWIl to INSRWI32 - BRR replicated weights 1-32 (final institution weight - 
teacher preparation unit) 

• FINRWIl to FINRWI32 - BRR replicated weights 1-32 (final teacher preparation 
unit weight) 

• INSWGTI - Final institution weight - teacher preparation unit 

• FINWGTI - Final teacher preparation unit weight 

• INSRWFl to INSRWF32 - BRR replicated weights 1-32 (final institution weight - 
educator) 

• FINRWFl to FINRWF32 - BRR replicated weights 1-32 (final educator weight) 

• INSWGTE - Final institution weight (educator) 

• FINWGTE - Final educator weight 

• INSRWPl to INSRWP32 - BRR replicated weights 1-32 (final institution weight - 
future primary teacher) 

• FINRWPl to FINRWP32 - BRR replicated weights 1-32 final future teacher weight 
(future primary teacher) 

• INSWGTP - Final institution weight (future primary teacher) 

• FINWGTP - Final future teacher weight (future primary teacher) 

• INSRWSl to INSRWS32 - BRR replicated weights 1-32 (final institution weight - 
future lower-secondary teacher) 

• FINRWSl to FINRWS32 - BRR replicated weights 1-32 final future teacher weight 
(future lower-secondary teacher) 

• INSWGTS - Final institution weight (future lower-secondary teacher) 

• FINWGTS - Final future teacher weight (future lower-secondary teacher) 

• TPUPROGR - Program name of teacher preparation unit 

• TPUPNUM - Program type number 

• TPURNUM - Route number of teacher preparation unit 

• INTED08 - Indicates if record is part of the international database 

2.3. TEDS-M Within-Country Scoring Reliability Data Files 

The TEDS-M within-country scoring reliability data files contain data that can be used 
to investigate the reliability of the TEDS-M constructed-response item scoring. The 
scoring reliability data files contain one record for each booklet that was double scored 
during the within-country scoring reliability exercise. For each constructed-response 
item in the Part C of the future teacher questionnaire, the following three variables are 
included in the scoring reliability data files: 

• Original score: two-digit score assigned by the first scorer 

• Second score: two-digit score assigned by the second scorer 

• Score agreement: degree of agreement between the two scorers 
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It should be noted that the second score data were used only to evaluate within- country 
scoring reliability and were not used in computing the achievement scores included in 
the database and presented in the international reports. 

2.3.1. Scoring Reliability Variable Naming Convention 

The variable names for the original score, second score, and score agreement variables 
are based on the same naming convention as for the achievement item variables shown 
earlier. The first character in the variable name differentiates between the three reliability 
variables: 

• The original score variable has the letter “M” as the first character, in accordance with 
the achievement item naming convention (e.g., MFC105B) 

• The second score variable has the letter “R” as the first character (e.g., RFC105B) 

• The score agreement variable has the letter “I” as the first character (e.g., IFC105B). 

2.3.2. Scoring Reliability Variable Score Values 

The values contained in both the original score and second score variables are the two- 
digit diagnostic codes assigned using the TEDS-M scoring guides. The score agreement 
variable may have one of three values, depending on the degree of agreement between 
the two scorers. 

• Code 0: Identical codes (both digits in the original and second scores) 

• Code 1: Identical score levels, but different diagnostic codes (first digit of both 

scores are the same; second digits are different) 

• Code 2: Different score levels (first digit of both scores are different) 

2.4. TEDS-M Codebook Files 

All information related to the structure of the TEDS-M data files, as well as the source, 
format, descriptive labels, and response option codes for all variables is contained in the 
codebook files. Each data file type in the database is accompanied by a codebook file. 
The codebooks are provided in “^.sdb (structural database) and *.pdf format. 

2.4.1. dBase Format (*.SDB) 

These dBase style files contain the information from the codebook as provided to the 
NRCs for data entry with the lEA Windows Data Entry Manager (WinDEM) software. 
After data processing, all added or changed variables are reflected in these final 
codebooks. This format can be read for instance with dBase, MS Access, or WinDEM. 
The abbreviation SDB stands for Structural Data Base. 

The following final main study codebook files are included in the TEDS-M international 
database: 

• DIGTEDTl.SDB: Institutional program codebook file 

• DEGTEDTl.SDB: Educator questionnaire codebook file 

• DPGTEDTl.SDB: Euture primary teacher codebook file 

• DPRTEDTl.SDB: Euture primary teacher reliability codebook file 

• DSGTEDTl.SDB: Euture lower-secondary teacher codebook file 

• DSRTEDTl.SDB: Euture lower- secondary teacher reliability codebook file 
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2.4.2. Portable Document Format (*.PDF) 

The portable document format (PDF) is a file containing the information from the 
codebook in a printout format. This format can be read with the freely available Acrobat 
Reader® or other PDF viewers. The information for each variable is presented in several 
lines of text. The lines for each variable are properly labeled. 

The following codebook files are provided: 

• DIGTEDT1.PDF: Institutional program codebook file 

• DEGTEDT1.PDF: Educator questionnaire codebook file 

• DPGTEDTl.PDE: Future primary teachers codebook file 

• DPRTEDT1.PDF: Future primary teacher reliability codebook file 

• DSGTEDT1.PDF: Future lower- secondary teachers codebook file 

• DSRTEDT1.PDF: Future lower-secondary teacher reliability codebook file 

2.5. TEDS-M Data Almanac Files 

The almanacs provide weighted summary statistics for each program group and the 
participating country where programs exist in that group on each individual item 
included in the TEDS-M assessments. There are separate almanacs for assessment items 
and questions, which are further separated by file type. There are a total of six files: 

• TEDS-M_ltemAlmanac_DPG.pdf 

• TEDS-M_ltemAlmanac_DSG.pdf 

• TEDS-M_QuestionAlmanac_DlG.pdf 

• TEDS-M_QuestionAlmanac_DEG.pdf 

• TEDS-M_QuestionAlmanac_DPG.pdf 

• TEDS-M_QuestionAlmanac_DSG.pdf 

The almanacs also display the international averages for each item within a program 
group, with each applicable country weighted equally. 

2.5.1. TEDS-M Knowledge Data Almanac Files 

The knowledge data almanacs display the content and cognitive domains of each item, 
the item block it belongs to, a brief description of the item, its variable name, whether 
it is a multiple-choice or constructed-response item type, and the correct response key 
if it is a multiple- choice item. 

There are two types of displays in the achievement data almanacs, depending on 
whether an item is a multiple-choice item or a constructed-response item. The statistics 
displayed in these almanacs are as follows: 

• N: The number of future teachers to whom the survey was administered. 

• Valid N: The number of future teachers to whom the survey was administered minus 
the number of future teachers to whom the specific item was not administered. 

• DIEE: Percentage of future teachers who responded correctly to a multiple-choice 
item. 

• 1-4: The percentage of future teachers choosing each one of the response options of 
a multiple-choice item (up to 4 response options). 

• Scoring Guide Codes (10, 11, 21, 22, 70, 71, etc.): The percentage of future teacher 
responses assigned to each of the codes in the scoring guide for a constructed- 
response item. 
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• OMITTED: The percentage of future teachers who omitted to respond to the item. 

• NOT REACHED: The percentage of future teachers who did not reach the item (i.e., 
they did not proceed to the item because the session time was over). 

• VI, V2: The percentage of future teachers who scored 1 point (VI) or 2 points (V2) 
on the item. 

2.5.2. TEDS-M Question Data Almanac Files 

The question data almanacs display the distribution of responses per country to each of 
the questions, separate by program group. 

• N: The number of future teachers to whom the survey was administered. 

• Valid N: The number of future teachers to whom the survey was administered minus 
the number of future teachers to whom the specific question was not administered. 

• MEAN, STDDEV, MODE: These columns correspond to the mean, standard 

deviation, and mode of the distribution of responses to the question. If the variable 
has multiple modes, the cell reads 

• MIN, P5, PIO, QI, MEDIAN, Q3, P90, P95, MAX: In case of a non-categorical 
question, information on the variable values at certain points of the distribution 
is presented. The columns correspond to the minimum point, the 5th, 10th, 25th, 
50th, 75th, 90th, 95th percentiles, and the maximum point in the distribution of 
responses. 

• [CATEGORIES - VALUE LABELS]: For categorical questions, the percentages of 
respondents per category are shown. The column headers display the value label of 
all possible answer categories in the order of the values assigned to the categories. 

• NOT APPLICABLE: The percentage of observations with “logically not applicable” 
codes. 

• NOT ADMINISTERED: The percentage of observations with “not administered” 
codes. 

• OMITTED: The percentage of observations with “omitted” codes. 

2.6. TEDS-M Syntax Files 

The TEDS-M International Database contains SPSS syntax files to perform variable 
recodes required for the proper execution of example analyses using the lEA IDB 
Analyzer. They are described in Chapter 4 of this TEDS-M IDB User Guide. In addition, 
the file SYNTAX_for_International_Report_OTL_Data.sps and its accompanying 
READ ME text file provide syntax and instructions to recode the OTL data according to 
the results reported in the TEDS-M International Report (Tatto et ah, 2012). All these 
SPSS programs are part of the TEDS-M International Database available on the lEA 
Study Data Repository webpage at http://rms.iea-dpc.org/. 
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CHAPTERS 

Sampling Weights and Variance Estimation in 
TEDS-M 

3.1. Sampling Weights 

3.1.1. Why Weights are Needed 

The TEDS-M study includes populations of future primary and secondary school 
mathematics teachers within the participating countries, their educators, and their 
institutions. The data comprised in the international database are obtained from random 
representative samples or censuses of these populations.' The estimation weights reflect 
the complex sample designs that were applied in this study. Estimation weights have to 
be used for data analysis in order to obtain design-unbiased estimates of population 
features, instead of merely reflecting sample data (Lohr, 1999). 

Estimation weights reflect and compensate the different selection probabilities and the 
different non-response patterns at the various sampling stages. All base weights and 
non-response adjustment factors were computed separately for each of the four target 
populations and result in four different final (or “estimation”) weights. 

Response units with small selection probabilities received large weights, and, conversely, 
response units with large selection probabilities received small weights. To give an 
example, in countries where all institutions were asked to take part, the selection 
probability at this first sampling stage was equal to one. Then, within the small 
institutions, all future teachers were asked to participate. Thus, the selection probability 
at this second sampling stage was again one. In large institutions however, only 30 
future teachers were sampled. Therefore, these individuals had selection probabilities 
below one. This fact is accounted for by assigning larger weights to future teachers in 
larger institutions. In more specific terms, the base weight is calculated as the inverse of 
the selection probability. 

Given that in some sampled institutions, future teachers and educators refused or 
were unable to participate in the study, it was necessary to adjust the base weights to 
compensate for the sample size loss. Thus, the base weights were multiplied by non- 
response adjustments. The final (estimation) weights are the product of all weight and 
adjustment factors. They display how many units in the population are represented 
by each participating unit. Consequently, the overall sum of the final weights of all 
respondents is an estimation of the size of the corresponding population. Weighting each 
respondent by its final weight in all analyses ensures that the respondent contributes to 
the analyses in accordance with the number of units in the population it represents. 

Details about the general sampling design are reported in Chapter 7 of the TEDS-M 
Technical Report (Dumais & Meinck, in Tatto [2012 a]). Details about national 
implementation can be found in the Appendix entitled Characteristics of National 


1 Even in the cases where full censuses of the respective populations were intended, not all sampled institutions or 
targeted individuals agreed to participate in the survey. Further, as a basis for scientific generalization, censuses 
can be seen as random samples from superpopulations. The concept of superpopulations was first described by 
Deming & Stephan (1941), and is applied in all lEA surveys. 
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Samples: Implementation of the International Sampling Design in Participating Countries 
of the TEDS-M Technical Report (Meinck & Dumais, in Tatto [2012]). All procedures 
to calculate weights are explained in Chapter 11 of the TEDS-M Technical Report 
(Dumais & Meinck, in Tatto [2012 b]). 

3.1.2. Selecting the Appropriate Weight Variable 

Each database is supplied with the appropriate set of weights. 

• For analyses of the institutional program questionnaire (institutional program files, 
DIG), the final teacher preparation uniC (TPU) weight FlNWGTl must be used. 

• For analyses of the educator questionnaire (educator file, DEG), the final educator 
weight FINWGTE must be used. 

• For analyses of data collected from future primary teachers (DPG file), the final 
future teacher weight (primary) FINWGTP must be used. 

• For analyses of data collected from future lower-secondary^ teachers (DSG file), the 
final future teacher weight (secondary) FINWGTS must be used. 

Moreover, each file type (DIG, DEG, DPG, and DSG) will contain a final institution 
weight (INSWGTI, INSWGTE, INSWGTP, and INSWGTS). For a given institution, this 
final institution weight may differ between the different file types. This difference is 
due to the fact that an institution can count as having participated in, say, the survey of 
future primary teachers, but not in the survey of educators. In such cases, non-response 
adjustments at the institutional level differ with the surveyed population. The final 
institution weight has to be applied if a file is aggregated by institution (or by TPU in 
the case of the two future teacher populations) for further analysis. 

If researchers plan to analyze data from more than one target population, they must 
carefully choose the correct weight. 

• Analyzing combined data from future primary (or lower-secondary) teachers 
and data from the institutional program files is straightforward with the lEA IDE 
Analyzer (see Chapter 4 of this user guide). The software can be used to easily add 
program level data to future teacher data. In this way, program information becomes 
an attribute of future teachers, and the user can analyze information from both files. 
For example, the research question could be of the type: “What percentage of future 
primary teachers attends programs with a particular attribute?” FINWGTP must be 
used for this type of data analysis. The use of FINWGTI is inappropriate in this case 
and the lEA IDE Analyzer automatically chooses the correct weight. 

• If future teacher information is regarded as an attribute of the program information, 
for example, the research questions could state: “What percentage of programs with a 
particular characteristic contain no/ any/ mostly/only future teachers with a particular 
characteristic?” This analysis cannot be performed with the lEA IDE Analyzer. The 
researcher must use other software (e.g., SPSS, SAS) first to aggregate the future 
teacher data and then to merge the future teacher information to the institutional 
program file. When statements are made about programs that have aggregated future 
teacher information attached, FINWGTI must be used. 


2 Institutions were separated by the different programs they offered. For the sake of clarity, these programs were 
referred to as teacher preparation units (TPU). All in-scope future teachers were allocated to exactly one TPU 
within their institutions. See Dumais & Meinck (in Tatto, 2012) for more information. 

3 Note that some surveyed future teachers were aiming for a certificate to teach lower-secondary and upper 
secondary students (see program grouping in Chapter 1.3.1). 
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• If educator information is regarded as an attribute of the program information, for 
example, the research questions could state: “What percentage of programs with 
a particular characteristic employ no/any/mostly/only educators with a particular 
characteristic?” This analysis also cannot be performed with the lEA IDE Analyzer. 
The researcher must use other software (e.g., SPSS, SAS) first to aggregate the 
educator data^ and then to merge the educator information to the institutional 
program file. When statements are made about programs that have aggregated 
educator information attached, FINWGTI must he used. 

It should he noted that any annotation referring to either of the combined populations 
will then also apply to the combined file. Furthermore, potential instances of missing 
data due to the combination of two file types may jeopardize the validity of any 
results. 

It is neither possible nor meaningful to combine other file types for various reasons. 

• Future primary teacher files cannot be combined with future lower-secondary 
teacher files because: 

- The future teachers constitute two separate target populations and received 
different assessments; 

- The two populations partially or fully overlap in some participating countries^, 
and the final weights account for this overlap. Combining the two files would 
lead to a severe overestimation of the total population size of future primary and 
lower-secondary teachers in the particular countries, as well as introduce bias into 
any other estimates obtained from such a combined file. 

• Future teacher files cannot be combined with educator files because educators and 
future teachers are neither conceptually nor organizationally linked to each other. In 
fact, educators participating in the survey may never have taught the participating 
future teachers. 

3.1.3. Example for Analyzing Weighted Data 

Not using weights in data analysis can lead to severely biased results. The following 
example illustrates the importance of using weights in research with TEDS-M data. 

A researcher may be interested in finding out whether there were any circumstances 
that hindered future teachers during their teacher preparation program. For example, 
consider primary mathematics specialists (DPG file, variable TARGET? = 4) in 
Germany. The analysis variables are MFA012A, MFA012B, and MFA012G. The results 
of this analysis, using unweighted vs. weighted data, are displayed in Table 3.1. Let us 
focus on the variable MFA012A. Neglecting the different selection probabilities, one 
might think that almost every third future teacher in the population of interest had 
family responsibilities (29.8%), while in actual fact, this is only the case for about every 
fifth student (18.7%) once the selection probabilities are taken into account and the 
data are weighted. 


4 Aggregating educator information to the program level is not straightforward. An educator could teach in 
different programs in a given institution. The string variable TPUIDS in the educator file contains all program 
identification numbers (the corresponding identification number in the institutional program file is IDTPU), 
separated by commas, for any program an educator stated he or she taught during the survey administration 
period. 

5 Chile, Germany, Norway, Poland, Thailand, United States. 
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Table 3.1: Example of Incorrect (Unweighted) and Correct (Weighted) Analyses 


Variable 


Unweighted, % 
(Biased Estimate) 

Weighted, % 
(Unbiased Estimate) 

MFA012A 

Yes 

29.8 

18.7 

"Had family responsibilities that 
made it difficult to do my best" 

No 

70.2 

81.3 

MFA012B 

Yes 

18.3 

23.3 

"Had to borrow money" 

No 

81.7 

76.7 

MFA012C 

Yes 

51.1 

45.5 

"Had to work at a job" 

No 

48.9 

54.5 


3.2. Variance Estimation 

3.2.1. Why Variance Estimation is Needed 

All estimates, statements, and inferences in TEDS-M are based on data derived from 
random samples. Randomness is naturally associated with a certain risk of “error”. The 
sampling variance (or the sampling error as the square root of the sampling variance) is 
a measure of the uncertainty due to random sampling. Every generalization made from 
a sample should be reported along with its sampling error in order to make the reader 
aware of the precision of the result. 

Due to the complex sampling design applied in TEDS-M, it is not possible to correctly 
calculate sampling errors or to perform significance tests that yield correct results with 
standard procedures available in software packages such as SPSS. These programs 
assume that the data are derived from simple random samples. But this is not the case 
for TEDS-M data. Instead, various sampling designs were applied in participating 
countries (see Meinck & Dumais, in Tatto [2012], for more information). Some 
countries implemented multistage stratified sample designs. Such sampling designs, 
being convenient and cost-effective on the one hand, are known to be less efficient 
in terms of precision, on the other hand (Kish, 1987). In other words, compared to a 
simple random sample with equal sample size, the sampling variance in cluster samples 
tends to be higher. In other countries however, stratified simple random samples were 
selected. In these cases, the precision of the estimates can be even higher compared 
to a simple random sampling design. This is the case if the stratification variables are 
correlated with the variables of interest. 

A variance estimation method called “balanced repeated replication” (BRR), more 
specifically Bay’s variation of BRR (Bay, 1989; Judkins, 1990; Lohr, 1999), needs to be 
applied when estimating variances of TEDS-M data. This method takes into account 
the complexity of the sample designs and will deliver (approximately) design-unbiased 
variance estimates (Wolter, 1985). The international database contains variables that 
allow for the use of this variance estimation method. These variables are referred to 
as BRR replicate weights. The BRR technique has been implemented in the lEA IDB 
Analyzer software (lEA, 2010). This method of variance estimation is also recognized 
by other software packages.® For details about the BRR technique used for TEDS-M, 


6 WesVar (Westat Inc., 2008), SAS 9.2 (SAS Institute Inc., 2008), SUDAAN 10 (RTI, 2008), and Stata 10 (StataCorp 
LP, 2009) 
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i.e. how replicate weights were computed, how the sampling variance was computed, 
and how estimates can he compared, please refer to Chapter 1 1 of the Technical Report 
(Dumais & Meinck, in Tatto [2012 h]). 

3.2.2. Selecting the Appropriate Variance Estimation Variables 

The procedure to select the appropriate variance estimation variables follows. 

A set of replicate weights were calculated 

• Separately for each participating country and each target population; 

• Separately for both final weights (final population and final institution weight, see 
Section 3.2). 

Thirty-two replicate weights per set were calculated, and form part of the data files; 
these replicate weights must be used for variance estimation. The “zone” and “replicate” 
markers which users of other large-scale surveys may be expecting are not required for 
the TEDS-M international database. 

• For analyses of the institutional program questionnaire (institutional program files, 
DIG), the replicate weights FlNRWll to F1NRW132 must be used. 

• For analyses of the educator questionnaire (educator file, DEG), the replicate weights 
FlNRWEl to F1NRWE32 must be used. 

• For analyses of data collected from future primary teachers (DPG file), fhe replicate 
weights FlNRWPl to F1NRWP32 must be used. 

• For analyses of data collected from future lower- secondary teachers (DSG file), the 
replicate weights FINRWSl to F1NRWS32 must be used. 

The lEA IDB Analyzer makes use of these replicate weights for variance estimation by 
default. 

If data is aggregated by institution (or by TPU in case of the two future teacher 
populations) and will be used in this format for further analysis, the replicate weights 
corresponding to the respective final institution weight must be applied. They are 
labeled as follows: 

• Institutional program files (DIG): INSRWll to INSRWI32 

• Educator files (DEG): INSRWEl to 1NSRWE32 

• Future primary teachers files (DPG): INSRWPl to INSRWP32 

• Future lower- secondary teachers files (DSG): INSRWSl to 1NSRWS32. 

Consequently, each file contains 64 replicate weight variables in total. 

3.2.3. Example for Variance Estimation 

Failing to apply the BRR method may lead to severe bias in the estimation of sampling 
error. In the following example, sampling error estimates incorrectly assuming simple 
random sampling are compared with sampling error estimates that correctly consider 
the complex sample design through the use of BRR. The differences should illustrate 
the importance of using the correct variance estimation method for any analysis. 

For instance researchers might be interested in the average mathematics content 
knowledge (variable MCK) of lower-primary generalist teachers. Table 3.2 presents 
the results for three countries. The difference between the correct (unbiased) and the 
incorrect (biased) estimate is rather small in Georgia and Switzerland - these two 
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countries conducted censuses of institutions and applied simple random sampling 
within institutions for this particular future teacher cohort. However, the unbiased 
sampling error estimate for Russia is five times as large as the biased one because Russia 
selected a clustered sample. The main reason for the increased sampling error is the fact 
that future teachers belonging to the same institution are more likely to have similar 
mathematics content knowledge than future teachers from different institutions. This 
effect is taken into account when applying BRR for variance estimation. 


Table 3.2: Example of Incorrect and Correct Variance Estimation 


MCK (Mathematics 
Content Knowledge) 
Lower-Primary Generalists 
(Grade 4 maximum) 

Mean 

(Weighted) 

S.E. (Mean) 
(Biased Estimate, 
Assumption: 
SRS Sampling) 

S.E. (Mean) 
(Unbiased Estimate 
Derived from BRR) 

Georgia 

344.7 

3.80 

3.85 

Switzerland 

512.2 

5.76 

6.43 

Russia 

535.5 

1.92 

9.89 


One may also be interested in cross-country comparisons. It can be seen that the 
performance of lower-primary generalists in Georgia is lower than the corresponding 
performance in Switzerland or Russia. However, quite a substantial difference in the 
variable of interest seems to exist also between Switzerland and Russia (23.3 points). 
The t-test, performed in SPSS (computing the sampling error of the difference between 
the two means under the [incorrect] assumption of SRS), gives the incorrect result that 
there is a statistically significant difference (p=0.2%)^. If the estimates are compared 
accounting for designs and weights, then there is no statistically significant difference 
(p=5%). 


7 More precisely, 99.8% of all possible samples would also detect a difference between the two countries. 
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CHAPTER 4 

Using the lEA IDB Analyzer to Analyze the 
TEDS-M International Database 

4.1. Overview 

This chapter describes the use of the lEA International Database (IDB) Analyzer 
software (lEA, 2011) for analyzing the TEDS-M international data files. Example 
analyses will illustrate the capabilities of the lEA IDB Analyzer to compute a variety 
of statistics, including percentages of future teachers in specified subgroups, average 
mathematics content knowledge in these subgroups, as well as correlations and 
regression coefficients. The examples use data from the future teacher, educator, and 
institutional program questionnaires to replicate some of the TEDS-M results included 
in the TEDS-M International Report (Tatto et al., 2012), as well as other useful analyses 
for investigating policy-relevant research questions. 

The lEA IDB Analyzer uses the SPSS data files as well as the SPSS syntax files (Syntax_ 
MFA002.SPS and Recode_MIC002B.SPS) that are printed in the corresponding section 
to recode variables used in an example regression analysis presented later in this 
chapter. 

4.2. The lEA IDB Analyzer 

Developed by the lEA Data Processing and Research Center (lEA DPC), the lEA IDB 
Analyzer software uses the Statistical Package for the Social Sciences (SPSS, 2011) as an 
engine for statistical computations. The lEA IDB Analyzer enables users to combine SPSS 
data files from lEA’s large-scale assessments and conduct analyses using SPSS without 
actually writing programming code. The lEA IDB Analyzer generates SPSS syntax 
that takes into account information from the sampling design for the computation of 
statistics and their standard errors. 

The lEA IDB Analyzer consists of two modules — the merge module and the analysis 
module, which are executed as independent applications. The merge module is used to 
create datasets for analysis by combining data files of different types and from different 
countries, and selecting subsets of variables for analysis. The analysis module provides 
procedures for computing various statistics and their standard errors for variables of 
interest. These procedures can be applied for a country and for specific subgroups within 
a country. Both modules can be accessed through the START menu in Windows: 

Start All Programs c(> lEA IDB Analyzer c|> Merge Module / Analysis Module 

4.3. Merging Files with the lEA IDB Analyzer 

The TEDS-M data files are stored in the database separately for each country and by file 
type as described in Chapter 2. In addition to allowing users to combine datasets from 
similar respondent populations from more than one country for cross-country analyses, 
the merge module allows for the combination of data from different sources. However, 
due to the study design, the lEA IDB Analyzer supports only two combinations of 
TEDS-M data files - institutional program data and future primary teacher data (DIG 
and DPG), and institutional program data and future lower-secondary teacher (DIG 
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and DSG) data files. Other combinations of the TEDS-M file types would violate the 
study design and are thus not supported by the lEA IDB Analyzer. Euture teacher data 
cannot be merged with educator data directly, because there is no direct link between 
the two. Both populations were sampled independently of each other. Even if educators 
were from the same institutions, some might never have taught the future teachers who 
were surveyed. Merging institutional program and educator data is also not possible 
with the lEA IDB Analyzer. In most countries, some educators teach in more than one 
program or even institution (see Tatto et al., 2012). Eor reference, the variable TPUIDS 
in the educator data files lists the programs educators teach in with their program 
identification number (IDTPU). 

4.3.1. Merging Future Primary or Lower-Secondary Teacher and Institutional 
Program Data 

This IDB user guide includes a detailed description of the procedure for merging future 
teacher and institutional program data files only. As explained above, the future teacher 
and program data represent the single straightforward merge of data from different 
levels in TEDS-M. 

Eor the examples that follow all countries participating in TEDS-M will be merged and 
included in the analysis. Data from Norway does not permit national-level analyses, 
however. Norwegian program types are reported separately because the populations 
partly overlap and therefore data from these program types cannot be aggregated. This 
is why in all analysis outputs Norway will appear as Norway (ALU), Norway (ALU -t), 
and Norway (PPU & MASTERS) for each of the different program types available. 

The following steps will create an SPSS data file with data from multiple countries and 
multiple file types. In this example we will merge future primary teacher data with 
institutional program data. Merging the future lower-secondary teacher data with 
institutional program data follows the same logic. 

1 ) Open the Merge Module of the lEA IDB Analyzer. 

2) In the Select Data Directory field, browse to the folder where all SPSS data files 
are located. Eor example, in Eigure 4.1, all SPSS data files are located in the “C:\ 
TEDS2008\Data” folder. The program will automatically recognize and complete 
the Study Type and Survey Type fields and list all countries available in this folder 
as possible candidates for merging. If the folder contains data from more than one 
TEA study, or from more than one population and cycle, the lEA IDB Analyzer will 
prompt users to select files from the desired study and population for analyses. In 
Figure 4.1, the TEDS-M study is selected. 

3 ) Select the countries of interest from the Available Participants list. To select multiple 
countries, hold the CTRL key of the keyboard when selecting the countries and 
then press the single-arrow button (►) to move them in the Selected Participants 
list on the right. In the current example all countries participating in TEDS-M 
are selected for merging. Simply press the double-arrow button (►►) to select all 
available data. Figure 4.1 shows the lEA IDB Analyzer screen after selecting all 
countries for merging. 
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Figure 4. 1 lEA IDB Analyzer Merge Module: Selecting Countries 



4) Press the Next» button to proceed. The software will open the second window of 
the merge module, as shown in Figure 4.2, to select the file types and the variables 
to be included in the merged data file. 

5) Select the File Types (in this case the Future Teacher File — Primary and the 
Institutional Program File) for merging by checking the appropriate boxes to the 
left of the window. 

6) For each file type, select the desired variables from the list of background variables 
available in the left panel. You can select and move separate variables from the 
Available Variables to the Selected Variables list by holding the control key, 
pressing the left mouse button, and then clicking the arrow button (►). If you 
want to select all variables and move them to the Selected Variables list, use the 
double-arrow key (►►). In our example we will select all variables from both the 
Institutional Program File and the Future Teacher File — Primary for merging. 
Please note that all identification and sampling variables are selected automatically 
by the lEA IDB Analyzer. 

7) Specify the desired name of the merged data file and the folder where it will be 
stored in the Output Files field. The lEA IDB Analyzer will create an SPSS syntax 
file (“^.SPS) with the same name and in the same folder with the code necessary to 
perform the merge. In the example shown in Eigure 4.2, the syntax file DIG&DPG- 
TEDS.SPS and the merged file DIG&DPG-TEDS.SAV will both be stored in the 
“C:\TEDS2008\Work” folder after running the syntax. The merged data file will 
contain both future primary teacher and institutional program data with the 
variables shown in the “Selected Variables” panel to the right. 
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Figure 4.2 lEA IDE Analyzer Merge Module: Selecting File Types and Variables 



8) Click on the Start SPSS button. The lEA IDE Analyzer will show a warning before 
overwriting an existing file with the same name in the specified folder. The lEA 
IDE Analyzer creates the SPSS syntax file with the specified name, sfores it in the 
specified folder, and opens it in an SPSS syntax window ready for execution. The 
syntax file must be executed by opening the Run menu of SPSS and selecting the 
All menu option as shown on Figure 4.3. 
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Figure 4.3 lEA IDB Analyzer Merge Module: Run SPSS Script for Merging Data 



Be sure to check the resulting SPSS output file for possible warnings. If warnings appear, 
check them carefully as they might indicate that the merge process was not performed 
properly and the resulting merged data file might not have been created as expected. 

4.3.2. Merging Data Files for the Examples 

To carry out the analysis examples that follow in this chapter, the following merged data 
files should be created with all available background variables and scores included: 

DIG&DPG-TEDS.SAV Merge the institutional program (DIG) and future primary 
teacher (DPG) data files from all countries 

DEG-TEDS.SAV Merge the educator (DEG) data files from all countries 

4.4. Performing Analysis with the lEA IDB Analyzer 

The analysis module of the lEA IDB Analyzer can be used to analyze any data files of 
the TEDS-M international database and all files created with the TEA IDB Analyzer 
merge module. Eor TEDS-M data the lEA IDB Analyzer analysis module can perform 
the following statistical procedures: 

Percentages and Means 

Computes percentages, means, and standard deviations for selected variables by 
subgroups defined by grouping variable(s) 

Percentages Only 

Computes percentages by subgroups defined by grouping variable(s) 

Regression 

Computes regression coefficients for selected variables to predict a dependent variable 
by subgroups defined by grouping variable(s) 
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Correlations 

Computes means, standard deviations, and correlation coefficients for selected variables 
by subgroups defined by grouping variable(s) 

All statistical procedures offered within the analysis module of the lEA IDE Analyzer 
make use of appropriate sampling weights. Standard errors are computed using the 
balanced repeated replication (ERR) with Fay’s modification for estimation of the 
variances (see Chapter 3 for more details). The Percentages and Means, Regression, 
and Correlations procedures can be performed either with background variables or 
knowledge scores. Knowledge scores are Rasch scores produced using Item Response 
Theory. See the TEDS-M Technical Report (Tatto, 2012) for details about the scaling 
procedures. 

To identify the variables for analysis the reader is advised to look either into the codebooks 
provided along with the data or in the questionnaires and released knowledge items 
contained in Supplements 1 and 4 respectively in this guide. 

For TEDS-M data the lEA IDE Analyzer requires the selection of the following variables, 
depending on the statistical procedure selected: 

Grouping Variables 

This is a list of variables to define subgroups. The list must consist of at least one grouping 
variable. Ey default, the lEA IDE Analyzer includes IDCNTRY. Eecause of the design 
of TEDS-M results regarding future teachers should be reported at the program group 
level only. Hence, the program group variable (TARGET? for future primary teacher 
data and TARGETS for future lower-secondary teacher data) should always be selected 
as a Grouping Variable in addition to the country identifier IDCNTRY. Additional 
variables may be selected from the available list. If the Exclude Missing from Analysis 
option is checked, only cases that have non-missing values in the grouping variables 
will be included in the analysis. 

Analysis Variables 

This is a list of variables for which means or percentages are to be computed, or the 
independent variables of a regression analysis. More than one analysis variable can be 
selected. 

Dependent Variable 

This is the variable to be used as the dependent variable of a regression analysis. Only 
one dependent variable can be listed. Dependent variables can be either contextual 
variables or knowledge scores. 

Weight Variable 

This variable accounts for different selection probabilities of the participants and will 
be used in the analysis. The lEA IDE Analyzer automatically selects the appropriate 
weight variable for analysis based on the file types included in the merged data file. 
FINWGTP (final future primary teacher weight) or FINWGTS (final future lower- 
secondary teacher weight) will be selected automatically when analyzing future primary 
or lower-secondary teacher data only, or with institutional program variables combined. 
The final institution weight (INSWGTI, INSWGTE, INSWGTP, and INSWGTS for 
the respective populations) has to be applied if a file is aggregated by institution (or 
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by TPU in the case of the two future teacher populations) and should serve in this 
format for further analysis. FINWGTI (Final Teacher Preparation Unit Weight) will 
be selected automatically when analyzing institutional program data only. FINWGTE 
(final educator weight) will be selected automatically when analyzing educator data (for 
more information on weights included in the different TEDS-M data sets and their use, 
please see Chapter 3). 

Replicate Weights 

BRR replicate weights are used for estimating the sampling variances (see Chapter 3). All 
32 replicate weights are available readily in the data files and are selected by the lEA IDB 
Analyzer automatically, depending on the weight variable, and cannot be changed. 

4.5. TEDS-M Analysis with Future Teacher Level Variables 

This section presents examples of analysis using the TEDS-M future teacher level data, 
including analysis used to produce tables from the TEDS-M International Report (Tatto 
et ah, 2012) as well as additional examples of analysis not included in the report. 

4.5.1. Percentages Only 

The first example will analyze whether future primary teachers from different teacher 
preparation programs had the opportunity to study functions as part of their teacher 
preparation. Since we want to report the proportion of future primary teachers (with 
their appropriate standard errors) that studied functions, relations, and equations as 
an area of their upper level of school mathematics in each program group, we will he 
computing percentages of future primary teachers for each of the mathematical topics 
and within each program group. 

We need to undertake a number of steps for this analysis. After reviewing the future 
primary teacher data codebook, we identify the variable MEB002D as the categorical 
variable containing the information on the opportunity to study functions. We also 
need to select the variable TARGET? as a grouping variable that identifies the primary 
teacher preparation program groups within the country. 

We will use one of the merged data files mentioned above for the analysis and perform 
the following steps using the analysis module of the lEA IDB Analyzer: 

1 ) Open the Analysis Module of the lEA IDB Analyzer. 

2) Select the merged data file DIG&DPG-TEDS.sav as the Analysis File. 

3 ) Select Percentages only as the Analysis Type. Note that there is one option available 
to check. Exclude Missing from Analysis, which should be checked. This option is 
checked by default to exclude cases with data missing for the grouping variables. 

4) The variable IDGNTRY is selected automatically as Grouping Variable. Additionally 
we need to add the variables TARGET? and MEB002D as Grouping Variables. 

5) The Weight Variable is automatically defined by the software. As this example 
analysis uses future primary teacher data, FINWGTP is selected automatically 
by default. The replicate weights EINRWPl to EINRWPl also are selected 
automatically, by default. 

6) Specify the name and folder of the output files in the Output Files field. The lEA 
IDB Analyzer will use this name and folder to create three output files: an SPSS 
syntax file that contains the code for performing the analysis, an SPSS data file with 
the results, and an Excel file with these same results. Set the number of decimals to 


zero. 
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7) Press the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Run menu of SPSS 
and selecting the All menu option. If the syntax file already exists, the lEA IDE 
Analyzer will prompt you to confirm overwriting the file. 

Figure 4.4 presents the screen of the analysis module with all selections completed for 
our example. 


Figure 4.4 lEA IDE Analyzer Setup for Example Future Teacher Level Percentages only 
Analysis 



The results for this analysis are displayed in Figure 4.5. The example shows each 
country’s proportion of future primary teachers per program group and each country’s 
proportion of sampled future primary teachers that reported studying different 
mathematics topics. The countries are identified in the first column as IDCNTRY is 
the first grouping variable. The second column represents the names of the different 
program groups (variable TARGETP) as the second grouping variable. The third 
column represents the answer categories of the third grouping variable, MFB002D. The 
fourth column reports the number of cases per country, program group and the future 
primary teachers’ answers. The fifth column reports the sum of weights of the sampled 
future primary teachers, followed by the percentage for each category accompanied by 
its ERR standard error. 

From the output presented in Figure 4.5 we can see that for the program group “Lower 
Primary Generalist (Grade 4 maximum)” the topic functions was studied by 91.8% of 
Russian future primary teachers. In Germany the proportion is 34.1% in that program 
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group. When interpreting the results the standard errors need to be taken into account. 
For example the standard errors for the “Primary Mathematics Specialist” program 
group in Germany (8.6%) are much higher than the standard errors for the “Lower 
Primary Generalist” program group in Germany (1.9%). The standard errors always 
need to be taken into account when comparing the results for the same program groups 
across countries if you want to test for statistical significance of the differences. In this 
case the reason for the large standard error for the “Primary Mathematics Specialists” 
in Germany is the small number of cases per group (“Primary Mathematics Specialist”, 
studied - n=55, not studied - n=42) and their population estimates (“Primary 
Mathematics Specialist”, studied - n=696, not studied - n=396). 
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Figure 4.5 Output for Example Future Primary Teacher Level Percentages only Analysis 


Percentages by 
‘COUNTRY ID* 

(IDCNTRY TARGET? MFB002D) 

‘LEVEL AT WHICH GRADUATES QUALIFY 
TO TEACH - PRIM* 

GEN \ STUDY 
MATH TOPIC 
SCH\FUNCTIONS 

N of 
Cases 

Sum of 
FINWGTP 

Percent 

Percent 
(s.e. ) 

Botswana 

PRIMARY/LOWER SECONDARY GENERALISTS 

STUDIED 

82 

95 

96.3 

2 . 2 


(GRADE 10 MAXIMUM) 

NOT STUDIED 

3 

4 

3 . 7 

2 . 2 

Chile 

PRIMARY/LOWER SECONDARY GENERALISTS 

STUDIED 

367 

1118 

55 . 8 

1 . 7 


(GRADE 10 MAXIMUM) 

NOT STUDIED 

285 

886 

44.2 

1 . 7 

Chinese Taipei 

PRIMARY GENERALISTS 

STUDIED 

590 

2313 

64.4 

1.3 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

332 

1279 

35 . 6 

1.3 

Georgia 

LOWER PRIMARY GENERALISTS 

STUDIED 

401 

522 

81 . 7 

1 . 7 


GRADE 4 MAXIMUM) 

NOT STUDIED 

89 

117 

18.3 

1 . 7 

Germany 

LOWER PRIMARY GENERALISTS 

STUDIED 

335 

1745 

34.1 

1.9 


(GRADE 4 MAXIMUM) 

NOT STUDIED 

590 

3378 

65 . 9 

1.9 


PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

55 

696 

63 . 7 

8.6 



NOT STUDIED 

42 

396 

36.3 

8.6 

Malaysia 

PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

387 

421 

67 . 7 

1.8 



NOT STUDIED 

184 

201 

32.3 

1.8 

Philippines 

PRIMARY GENERALISTS 

STUDIED 

560 

2761 

94.8 

2 . 2 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

29 

151 

5 . 2 

2 . 2 

Poland 

LOWER PRIMARY GENERALISTS 

STUDIED 

673 

1839 

38.5 

1.7 


(GRADE 4 MAXIMUM) 

NOT STUDIED 

1121 

2934 

61.5 

1.7 


PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

292 

1283 

97.7 

.7 



NOT STUDIED 

8 

31 

2.3 

.7 

Russian 

LOWER PRIMARY GENERALISTS 

STUDIED 

2051 

7823 

91 . 8 

1.0 

Federation 

(GRADE 4 MAXIMUM) 

NOT STUDIED 

203 

699 

8.2 

1.0 

Singapore 

PRIMARY GENERALISTS 

STUDIED 

149 

167 

56.5 

2.7 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

114 

129 

43.5 

2.7 


PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

60 

67 

52 . 0 

4.6 



NOT STUDIED 

57 

61 

48 . 0 

4.6 

Spain 

PRIMARY GENERALISTS 

STUDIED 

682 

2429 

63.3 

1 . 6 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

408 

1410 

36.7 

1 . 6 

Switzerland 

LOWER PRIMARY GENERALISTS 

STUDIED 

45 

66 

41 . 0 

4.2 


(GRADE 4 MAXIMUM) 

NOT STUDIED 

76 

94 

59.0 

4.2 


PRIMARY GENERALISTS 

STUDIED 

297 

394 

36.2 

1.5 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

514 

693 

63 . 8 

1.5 

Thailand 

PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

635 

1313 

96.5 

. 7 



NOT STUDIED 

23 

47 

3.5 

. 7 

United States 

PRIMARY GENERALISTS 

STUDIED 

954 

16634 

74.8 

2 . 1 


(GRADE 6 MAXIMUM) 

NOT STUDIED 

334 

5615 

25 . 2 

2 . 1 


PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 

154 

2919 

81.0 

4.0 



NOT STUDIED 

33 

685 

19.0 

4.0 

Norway (ALU) 

PRIMARY/LOWER SECONDARY GENERALISTS 

STUDIED 

380 

1393 

97.4 

. 8 


(GRADE 10 MAXIMUM) 

NOT STUDIED 

12 

37 

2 . 6 

. 8 

Norway (ALU +) 

PRIMARY/LOWER SECONDARY GENERALISTS 

STUDIED 

159 

433 

100.0 

. 0 


(GRADE 10 MAXIMUM) 






X. International 

LOWER PRIMARY GENERALISTS 

STUDIED 



17 . 9 

.3 

Average 

(GRADE 4 MAXIMUM) 

NOT STUDIED 



13.3 

.3 


PRIMARY GENERALISTS 

STUDIED 



24.4 

.3 


(GRADE 6 MAXIMUM) 

NOT STUDIED 



13 . 1 

.3 


PRIMARY/LOWER SECONDARY GENERALISTS 

STUDIED 



21 . 8 

. 2 


(GRADE 10 MAXIMUM) 

NOT STUDIED 



3 . 2 

. 2 


PRIMARY MATHEMATICS SPECIALISTS 

STUDIED 



28 . 7 

.7 



NOT STUDIED 



8.8 

.7 
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4.5.2. Percentages and Means 

In our second example, we want to replicate results presented in the TEDS-M 
International Report (Tatto et ah, 2012). This example examines the mean mathematics 
content knowledge of future primary teachers per program group per country. These 
results, presented in Table 5.10 of the TEDS-M International Report (Tatto et ah, 2012), 
are shown in Eigure 4.6 below. In this example the mathematics content knowledge 
(MCK) scores will be used. MCK is a single continuous variable for which we want to 
calculate the mean for each program group and country. 


Figure 4.6 Example of Percentage and Means with Achievement Scores Taken from the 
TEDS-M International Report (Exhibit 5.10) 


Perc»nia 9 « at Percentage at 
Percent or above or above Scaled 

Sample N^lld Data Missing AnchorPoInt Anchor Point Score: 

ProaramQrouo Country Size (M iWeiahiedi USF\ Mean fSf) 

Future Teachers (Primary) 
Mathematics Content Knowledge 

Group 1 
Lower Primary 
(Grade 4 Maximum) 

Georgia 

506 

506 

0.0 

11.9 (1.4) 

0.9 (0.5) 

345 (4) 









(Germany 

935 

907 

2.4 

86.4 (1.3) 

43.9 (2.1) 

501 (3) 









Poland * 

1812 

1799 

0.9 

67.9 (1.3) 

16.8 (1.2) 

466 (2) 










Russian Federation * 

2266 

2260 

0.2 

89.7 (2.3) 

57.3 (4.6) 

536 (10) 














Switzerland ' 

121 

121 

0.0 

90.5 (2.7) 

44.2 (5.4) 

512 (6) 









Groups 

Primary 

(Grade 6 Maximum) 

Chinees Taipei 

923 

923 

0.0 

99.4 (0.3) 

93.2 (1.4) 

623 (4) 












Philippines 

592 

592 

0.0 

60.7 (5.1) 

6.3 (0.9) 

440 (8) 














Singapore 

263 

262 

0.4 

100.0 

82.5 (2.3) 

586 (4) 












Spain 

1093 

1093 

0.0 

83.4 (1.6) 

26.2 (1.6) 

481 (3) 









Switzerland 

615 

815 

0.0 

97.2 (0.6) 

70.6 (1.4) 

548 (2) 









United Stales 

1310 

951 

28.6 

92.9 (1.2) 

50.0 (3.2) 

518 (5) 








Groups 

Primary.Secondary 
(Grade 10 Maximum) 

Botswana * 

86 

86 

0.0 

60.6 (5.3) 

7.1 (2.8) 

441 (6) 











Chile ' 

657 

654 

0.4 

39.5 (1.8) 

4.0 (0.7) 

413 (2) 














Norway (ALU) • 

392 

392 

0.0 

86.5 (1.5) 

46.9 (2.3) 

509 (3) 














Norway (ALU^) * 

159 

199 

0.0 

96.5 (1.4) 

68.7 (3.1) 

553 (4) 









Group 4 

Primary Specialists 

Germany 

97 

97 

0.0 

96.0 (2.1) 

71.7 (7.0) 

555 (8) 










lAalaysia 

576 

574 

0.4 

88.7 (1.1) 

28.1 (1.3) 

488 (2) 










Poland * 

300 

300 

0.0 

97.9 (1.0) 

91.0 (1.6) 

614 (5) 









Singapore 

117 

117 

0.0 

96.3 (1.2) 

87.3 (2.8) 

600 (6) 








Thailand 

660 

660 

0.0 

91.7 (0.9) 

56.2 (1.4) 

528 (2) 








United Stales 

191 

132 

33.2 

94.9 (1.7) 

48.1 (6.5) 

520 (7) 








Notes: 

tDenotes situations where data were available from fewer th^ 85% of respondents. 

When reading this table and chart, keep in mind the limitations listed earlter in this chapter. 

The shaded areas identify data that, for reasons explairted in the limitations, cannot be compared 
with confidence to data from other countries. 

The solid vertical lines on the chart show the two anchor points (431 and 516). 

Sin 

h 


2ai 7a*i 

T 

UB>ig>»dC<]'l'Ue'>c« l'l«'»alo2SC) 




With TEDS-M data we can use any continuous variable as an Analysis Variable in the 

Percentages and Means analysis procedure. Proceed as follows: 

1 ) Open the lEA IDB Analyzer analysis module 

2) Select the DIG&DPG-TEDS.sav data file as the Analysis File 

3) Select Percentages and Means as the Analysis Type. By default, the program will 
exclude records with missing grouping variables from the analysis. 

4) Add the variable TARGET? as a second Grouping Variable (IDGNTRY will be 
selected automatically). 

5) Specify the knowledge scores to be used for the analysis. Glick the Analysis Variable 
and select the variable MGK from the list of available variables and move it to the 
analysis variables field by clicking the right arrow button in this section. 

6) The Weight Variable is automatically identified by the software. As this example 
analysis uses future primary teacher data, FINWGTP is selected by default. 
The Replicate Weights variables FINRWPl to EINRWP32 also are selected 
automatically. 

7) Ghange the Number Of Decimals to 1 . 
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8) Specify the name and folder of the output files in the Output Files field. 

9) Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Run menu of SPSS 
and selecting the All menu option. If the syntax file already exists, the lEA IDB 
Analyzer will prompt you to confirm overwriting the file. 

Figure 4.7 below displays the analysis module with the proper settings for this example 
analysis. 


Figure 4. 7 lEA IDB Analyzer Setup for Example Percentages and Means Analysis 



The output for this setup is shown in Figure 4.8. In this example, each country’s results 
are presented on one or more lines depending on the number of available programs 
within the country. The countries are identified in the first column and the second 
column describes the category of TARGET? being reported. The third column reports 
the number of valid cases and the fourth the sum of weights of the sampled future 
primary teachers. The next two columns report the percentage of future primary 
teachers in each category (in this case each category represents a different program 
group) and its standard error, followed by the estimated mean mathematics knowledge 
and its standard error per program group. The standard deviation of the knowledge 
scores and its standard error are reported in the following two columns. The last column 
shows the percent of missing data. 
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Figure 4.8 Output for Example Future Primary Teacher Level Percentages and Means Analysis 


Average for MCK by (IDCNTRY TARGETP) 

‘LEVEL GRADS. QUAL 

‘COUNTRY ID‘ TO TEACH-PRIM* 

N of 
Gases 

Sum of 
FINWGTP 

Percent 

Percent 

(s.G.) 

MCK 

(Mean) 

MCK 

(s.G.) 

PAGE 

Std .Dev 
Std .Dev . (s.G.) 

1 

Percent 

Missing 

Botswana 

PRIMARY/LOWER SEC. GEN. 
(GRADE 10 MAX) 

86 

100 

100.0 

. 0 

441.2 

5.9 

48 . 1 

3.0 

. 0 

Chile 

PRIMARY/LOWER SEC. GEN. 
(GRADE 10 MAX) 

654 

2009 

100.0 

. 0 

413.0 

2 . 1 

64.9 

2.4 

.4 

Chinese Taipei 

PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

923 

3595 

100.0 

. 0 

623.2 

4.2 

84.2 

3.2 

. 0 

Georgia 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 

506 

659 

100.0 

. 0 

344.7 

3.9 

85 . 3 

3.4 

. 0 

Germany 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 

907 

5025 

82 . 1 

.3 

500.7 

2.9 

82 . 0 

3 . 7 

2.4 


PRIMARY MATHEMATICS 
SPECIALISTS 

97 

1093 

17.9 

.3 

555 . 2 

7.5 

73 . 9 

5.5 

. 0 

Malaysia 

PRIMARY MATHEMATICS 
SPECIALISTS 

574 

625 

100.0 

.0 

488 . 4 

1 . 8 

53.5 

4.0 

.4 

Philippines 

PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

592 

2921 

100.0 

. 0 

439 . 6 

7.6 

51 . 7 

3.6 

. 0 

Poland 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 

1799 

4788 

78.5 

. 2 

456 . 2 

2.3 

67.3 

2 . 2 

.9 


PRIMARY MATHEMATICS 
SPECIALISTS 

300 

1313 

21.5 

. 2 

614 . 2 

4.8 

92 . 2 

4.5 

. 0 

Russian 

Federation 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 

2260 

8545 

100.0 

.0 

535.5 

9.9 

91 . 1 

4.1 

. 2 

Singapore 

PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

262 

295 

69.7 

.5 

586.3 

3.7 

72.4 

3.8 

.4 


PRIMARY MATHEMATICS 
SPECIALISTS 

117 

128 

30.3 

.5 

599.6 

7.8 

76.1 

6.3 

.0 

Spain 

PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

1093 

3845 

100.0 

.0 

481.3 

2 . 6 

56.6 

1.6 

.0 

Switzerland 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 

121 

160 

12 . 8 

. 2 

512 . 2 

6.4 

62 . 8 

4.7 

.0 


PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

815 

1092 

87 . 2 

. 2 

547.9 

1.9 

65.0 

1.8 

.0 

Thailand 

PRIMARY MATHEMATICS 
SPECIALISTS 

660 

1364 

100.0 

.0 

528 . 1 

2.3 

75 . 1 

2.4 

.0 

United States 

PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 

951 

16159 

86.9 

5.3 

517.5 

4.5 

70.0 

4.1 

28 . 6 


PRIMARY MATHEMATICS 
SPECIALISTS 

132 

2433 

13 . 1 

5.3 

520.0 

6.6 

63 . 0 

3 . 8 

33.2 

Norway (ALU) 

PRIMARY/LOWER SEC. GEN. 
(GRADE 10 MAX) 

392 

1429 

100.0 

.0 

508.7 

3 . 6 

69.3 

2 . 8 

.0 

Norway (ALU +) 

PRIMARY/LOWER SEC. GEN. 
(GRADE 10 MAX) 

159 

433 

100.0 

.0 

552.8 

5.8 

74 . 0 

6.6 

.0 

X. International 
Average 

LOWER PRIMARY GEN. 
(GRADE 4 MAXIMUM) 



23.3 

.0 

469.9 

2 . 6 

77.7 

1 . 7 



PRIMARY GENERALISTS 
(GRADE 6 MAXIMUM) 



34.0 

. 3 

532.6 

1.8 

66 . 7 

1 . 3 



PRIMARY/LOWER SEC. GEN. 
(GRADE 10 MAX) 



25.0 

.0 

478.9 

2.3 

64 . 1 

2.0 



PRIMARY MATHEMATICS 
SPECIALISTS 



17 . 7 

. 3 

550.9 

2 . 3 

72.3 

1 . 9 
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4.5.3. Regression 

The lEA IDE Analyzer is able to calculate multiple linear regressions between dependent 
variables and a set of independent variables. This section demonstrates an example 
for a regression analysis with variables from the example merged data file DIG&DPG- 
TEDS.sav. 

With TEDS-M data the lEA IDE Analyzer can also be used to compute regression 
analysis with any variable included in the future teacher questionnaires or in the 
assessment such as the knowledge scores (MCK or MPCK). In this section a regression 
analysis will be performed using the mathematics content knowledge (variable MCK) 
as the dependent variable and future primary teachers’ gender (variable MFA002) as its 
predictor. The weighting variable FINWGTP will be selected automatically. 

For this example, variable MFA002 (future primary teacher gender) is recoded into a 
“dummy” variable REGMFA002 (with a value or “0” or “1” for ease of interpretation). 
This recoded variable can be created by running the SPSS syntax file Recode_MFA002. 
SPS. Figure 4.9 provides the syntax. Ey using this recoded variable, the intercept or 
constant will be the estimated average mathematics knowledge for females, whereas 
the regression coefficient (REGMFA002 [estimate]) will be the estimated difference 
in mathematics knowledge between females and males. This also allows a t-test to 
determine if the average mathematics knowledge is significantly different between 
females and males. After computing the new variable, the syntax file will save the data 
file with a new name: DIG&DPG-TEDS_REG.SAV. 

Figure 4.9 Example SPSS Program to Recode Variable MFA002for Future Primary Teacher 
Level Regression Analysis 

GET FILE = “C: \TEDS2008\Work\DIG&DPG-TEDS . sav” . 

* Gompute new variable REGMFA002 from MFA002. 

REGODE MFA002 (1=0) (2=1) (else=copy) INTO REGMFA002. 

EXECUTE . 

VALUE LABELS REGMFA002 0 ‘Female’ 

1 ‘Male’ . 

VARIABLE LABELS REGMFA002 "Recoded MFA002 (Female = 0; Male =1)”. 

SAVE OUTFILE = “C : \TEDS 2008 \Work\DIG&DPG- TEDS_REG . sav” . 


The example regression analysis is performed by the analysis module of the lEA IDE 

Analyzer with the completed analysis window shown in Figure 4. 10, using the following 

steps: 

1) Run the SPSS syntax file Recode_MFA002.SPS to create the variable REGMFA002 
and the file DIG&DPG-TEDS_REG.SAV. 

2) Open the Analysis Module of the lEA IDE Analyzer. 

3) Specify the data file DIG&DPG-TEDS_REG.SAV as the Analysis File. 

4) Select Regression as the Analysis Type. 

5) The variable IDCNTRY is selected automatically as a Grouping Variable. To reflect 
the differences between the programs, the program grouping variable for the future 
primary teacher data TARGETP needs to be added as a Grouping Variable. 
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6) Click on the Analysis Variables selection box to activate the section and select 
REGMFA002 as the analysis variable. This is done by selecting REGMFA002 from 
the list of available variables and moving it to the Analysis Variables field by 
clicking the right arrow button in this section. 

Figure 4.10 lEA IDB Analyzer Setup for Future Example Primary Teacher Level Regression 
Analysis 



7) Click on the Dependent Variables selection box and select the variable MCK 
from the list of available variables and move it to the Dependent Variable field by 
clicking the right arrow button in this section. 

8) The Weight Variable is automatically defined by the software. As this example 
analysis uses future primary teachers data, EINWGTP is selected by default. The 
Replicate Weights EINRWPl to FINRWP32 also are selected by default. 

9) Specify the name and folder of the output files in the Output Files field. 

10) Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Run menu of SPSS 
and selecting the All menu option. If necessary, the lEA IDB Analyzer will prompt 
you to confirm overwriting of already existing files. 

The results of this analysis are presented in Figure 4.11 where we can see that the 
differences between females and males for the “Primary Mathematics Specialist” program 
group in Poland are substantial (14.50 sore points), but also the standard errors for the 
constant and the slope are very large (7.76 and 23.97) which is a result of the very small 
number of cases in this program group - only 300. As a result we cannot claim that 
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there are statistically significant differences between female and male students in this 
program group in Poland even if differences exist. Germany has a much lower estimate 
of differences between females and males in both program groups (1.41 and 2.69) and 
the standard errors of the differences are much higher than the estimated differences 
themselves (5.16 and 5.64). Results for Botswana, United States, and Norway (ALU+) 
are similar. The large standard errors could be due to the small sample sizes. This is why 
the estimates should be always considered together with their standard error. Note that 
Georgia has an estimate of zero. This is due to the fact that only female future primary 
teachers were available in the Georgian sample and the variance is therefore equal to 
zero. Most of the countries show statistically significant results at a 95% confidence 
level since the t-test value is greater than 1.96 (two tailed). 
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4.5.4. Correlations 

The lEA IDB Analyzer is also able to compute correlations between two background 
variables, and between background variables and knowledge scores. As the TEDS-M 
scores are also single variables (and not a set of plausible values as in other lEA 
studies) the procedure for calculating correlations between two background variables 
and between background variable and knowledge scores is the same. The steps for 
conducting a correlation analysis are the same as those described previously: first select 
the grouping variables, then select the analysis variables (either one or two variables 
and/or the knowledge score). For each group defined by the grouping variables the 
output will display the correlation coefficients for each pair of variables. Note that the 
grouping variables are not displayed with their labels in the SPSS output, but the labels 
are displayed in the Excel output file and in the output SPSS system file. 

In this example, we correlate two derived background variables. Assume that for this 
example we are interested in the relationship between an opportunity to learn scale 
(mathematics education pedagogy - instructional practice) and a belief scale (beliefs 
about preparedness to teach mathematics). To perform this analysis, proceed as 
follows: 

1 ) Open the Analysis Module of the lEA IDB Analyzer. 

2) Specify the data file DIG&DPG-TEDS.SAV as the Analysis File. 

3) Select Correlations as the Analysis Type. By default, the program will exclude 
records with missing grouping variables from the analysis. 

4) The variable IDGNTRY is selected automatically as a Grouping Variable. To reflect 
the differences between the programs, the program grouping variable for the future 
primary teacher data TARGETP needs to be added as a Grouping Variable. 

5) Glick on the Analysis Variable and select variables MFD4PREP (the OTL scale) 
and MFB6IPRA (the belief scale) from the list of available variables and move it to 
the Analysis Variables field by clicking the right arrow button in this section. 

6) The Weight Variable is automatically defined by the software. As this example uses 
future primary teacher data, FINWGTP is selected by default. The replicate weights 
variables FINRWPl to FINRWP32 also are selected automatically. 

7) Specify the name and folder of the output files in the Output Files field. 

8) Glick the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Run menu of SPSS 
and selecting the All menu option. If the syntax file already exists, the lEA IDB 
Analyzer will prompt you to confirm overwriting it. 

Figure 4.12 below displays the analysis module with the proper settings for this example 
analysis. 
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Figure 4. 12 lEA IDE Analyzer Setup for Example Future Primary Teacher Level Correlations 
Analysis 


^ lEA IDB Analyzer Analysis Module 




O Analysis File:],; \TEDS2008\Woik\DIGSDPG TEDS_REG.!av 




““Sdect Analysis Type' 
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Modify 


O start SPSS I Exit | 


Help 


The output for this setup is shown in Figure 4.13. Each country’s results are presented in 
a separate table for each program group as defined by the grouping variable TARGETP. 
Please note that country labels are not displayed, only their values (for instance 
IDCNTRY=840 for the United States of America'). The countries are identified in 
the line above each table together with the program groups. The variable names are 
displayed in the first column. The sum of weights is displayed in the second column. 
The mean, the standard error of the mean, the standard deviation, and its standard 
error are displayed in the third to sixth columns. The remaining columns contain the 
correlation coefficients and their standard errors with the first two values in each row 
representing the correlation coefficient and the standard error of the variable correlated 
with itself (which are always 1 and 0 respectively). The two upper right numbers of the 
last two columns in each table contain the correlation estimate between the variables 
and its standard error. 

In Figure 4.13, we see that in all countries strong correlations exist between the 
instructional practice as an opportunity to learn and the belief of being prepared for 
teaching mathematics. The United States (IDCNTRY=840) show the highest correlation 
coefficient (0.61 with SE=0.06) and Germany (IDCNTRY=276) shows the lowest 
correlation coefficient (0.33 with SE=0.14) for the “Primary Mathematics Specialist” 
program group (TARGETP=4) among all countries. Note that the correlation coefficient 
for Germany is almost as twice as low, but the standard error is more than as twice as 
high compared to United States. 


1 See the country codes listed in Table 1 . 1 of Chapter 1 . 
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Figure 4.13 Output for Example Future Primary Teacher Level Correlations Analysis 


Correlation 

Variable 

matrix for: IDCNTRY= 72 

1 Sum of Wgts 1 Mean 

* TARGETP=3 * 
s.e. 1 StdDev 

S.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 
1 87.52 1 

12.75 

.182 

1.39 

. 125 

1.0000 .0000 

. 3977 

.1288 

MFB6IPRA 

1 87.52 1 

11.61 

. 135 

1.47 

. 105 


1 . 0000 

. 0000 

Correlation 

1 1 
matrix for: IDCNTRY= 152 

1 

* TARGETP=3 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 1884.53 1 

12.19 

.079 

1.99 

.051 

1.0000 .0000 

.5102 

.0404 

MFB6IPRA 

1 1884.53 1 

11.95 

.069 

1 .82 

.060 


1 . 0000 

. 0000 

Correlation 

1 1 
matrix for: IDCNTRY= 158 

1 

* TARGETP=2 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 3583.47 1 

10.74 

.044 

1.31 

.041 

1.0000 .0000 

. 4772 

.0283 

MFB6IPRA 

1 3583.47 1 

10.36 

.053 

1.62 

.041 


1 . 0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 268 

1 

* TARGETP=1 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 429.15 1 

11 . 00 

.084 

2 . 13 

.081 

1.0000 .0000 

. 4443 

.0553 

MFB6IPRA 

1 429.15 1 

10.03 

. 100 

1 .88 

.082 


1 . 0000 

. 0000 

Correlation 

1 1 
matrix for: IDCNTRY= 276 

1 

* TARGETP=1 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 4825.09 1 

10.12 

.084 

2.00 

.098 

1.0000 .0000 

. 5689 

.0435 

MFB6IPRA 

1 4825.09 1 

8.56 

.084 

2 . 12 

.065 


1 . 0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 276 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 1056.55 1 

10.54 

. 133 

1 . 12 

. 104 

1.0000 .0000 

. 3295 

. 1350 

MFB6IPRA 

1 1056.55 1 

9.51 

. 187 

1.38 

. 138 


1 . 0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 458 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 601.42 1 

12.63 

. 105 

2.01 

.079 

1.0000 .0000 

. 4605 

.0391 

MFB6IPRA 

1 601.42 1 

11.29 

.071 

1 .73 

.055 


1.0000 

. 0000 

Correlation 

1 1 
matrix for: IDCNTRY= 608 

1 

* TARGETP=2 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 2762.03 1 

12.92 

. 141 

1.57 

.060 

1.0000 .0000 

. 3554 

.0402 

MFB6IPRA 

1 2762.03 1 

12.11 

.188 

1.54 

.066 


1 . 0000 

. 0000 

Correlation 

1 1 
matrix for: IDCNTRY= 616 

1 

* TARGETP=1 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 - 
1 4615.52 1 

10.58 

.040 

1.75 

.045 

1.0000 .0000 

. 4278 

. 0254 

MFB6IPRA 

1 4615.52 1 

10.00 

.059 

1.88 

.057 


1 . 0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 616 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 . 
1 1283.72 1 

10.78 

.117 

1.63 

.119 

1.0000 .0000 

. 4423 

. 0465 

MFB6IPRA 

1 1283.72 1 

10.29 

.112 

1.66 

.064 


1 . 0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 643 

1 

* TARGETP=1 * 





Variable 

1 Sum of Wgts 1 

Mean 

s.e. 

StdDev 

s.e. 

Correlations and 

s.e. 


MFD4PREP 

1 1 _ 
1 8274.53 1 

11.76 

.083 

1.31 

.033 

1.0000 .0000 

.3718 

.0441 

MFB6IPRA 

1 8274.53 1 

10.86 

.118 

1 .78 

.048 


1 . 0000 

.0000 

II 1 1 
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Figure 4.13 Output for Example Future Primary Teacher Level Correlations Analysis (contd.) 


Correlation matrix for: IDCNTRY= 702 * TARGETP=2 * 


Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 r 

1 292.66 1 

11 . 68 

.080 

1.59 

. 096 

1.0000 .0000 

.4941 

.0548 

MFB6IPRA 

1 292.66 1 

11.28 

.093 

1.43 

.080 


1.0000 

.0000 

Correlation 

1 1 

matrix for: IDCNTRY= 702 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1 " 
1 128.00 1 

11 . 84 

. 167 

1.64 

. 107 

1.0000 .0000 

.3766 

.0798 

MFB6IPRA 

1 128.00 1 

11.09 

.091 

1 . 35 

.095 


1.0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 724 

1 

* TARGETP=2 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1 " 
1 3759.15 1 

11.25 

. 120 

2.06 

.065 

1.0000 .0000 

.4335 

.0616 

MFB6IPRA 

1 3759.15 1 

10.17 

. 189 

1 . 94 

. 117 


1.0000 

.0000 

Correlation 

1 1 

matrix for: IDCNTRY= 756 

1 

* TARGETP=1 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 r 

1 153.58 1 

11.29 

.118 

1.26 

.066 

1.0000 .0000 

.5309 

.0819 

MFB6IPRA 

1 153.58 1 

10.17 

.116 

1.26 

.076 


1.0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY= 756 

1 

* TARGETP=2 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 r 

1 1070.64 1 

10.82 

.037 

1.29 

.040 

1.0000 .0000 

.4708 

.0330 

MFB6IPRA 

1 1070.64 1 

9.94 

.037 

1.28 

.035 


1.0000 

.0000 

Correlation 

matrix for: IDCNTRY= 764 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1316.99 1 

12 . 12 

.045 

1.48 

.046 

1.0000 .0000 

.4859 

.0296 

MFB6IPRA 

1 1316.99 1 

11.36 

.077 

1.71 

.055 


1.0000 

.0000 

Correlation 

matrix for: IDCNTRY= 840 

1 

* TARGETP=2 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 17071.40 1 

12.54 

. 132 

1.99 

.054 

1.0000 .0000 

.5698 

.0337 

MFB6IPRA 

1 17071.40 1 

11.78 

. 136 

1.89 

.056 


1.0000 

.0000 

Correlation 

matrix for: IDCNTRY= 840 

1 

* TARGETP=4 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1 _ 
1 2697.84 1 

12 . 18 

. 185 

2.04 

.177 

1.0000 .0000 

. 6096 

.0581 

MFB6IPRA 

1 2697.84 1 

11.56 

.085 

1.67 

.097 


1.0000 

.0000 

Correlation 

1 1 

matrix for: IDCNTRY=57891 

1 

* TARGETP=3 * 





Variable 

1 Sum of Wgts 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1 " 
1 1387.41 1 

10.62 

.060 

1.46 

.069 

1.0000 .0000 

.4454 

.0500 

MFB6IPRA 

1 1387.41 1 

9.90 

.063 

1.38 

.069 


1.0000 

.0000 

Correlation 

1 1 
matrix for: IDCNTRY=57892 

1 

* TARGETP=3 * 





Variable 

1 Sum of Wgts 1 

Mean 

s . e . 

StdDev 

s . e . 

Correlations and 

s . e . 


MFD4PREP 

1 1 " 
1 400.90 1 

10.38 

.093 

1.22 

. 124 

1.0000 .0000 

.5422 

.0862 

MFB6IPRA 

1 400.90 1 

10.02 

.086 

1.23 

. Ill 


1.0000 

.0000 

II 1 1 
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4.6. TEDS-M Analysis with Educator Data 

For an example of an analysis using educator data, we will replicate Exhibit 4.6 from 
the TEDS-M International Report (Tatto et al., 2012), reproduced in Figure 4.14 below. 
We will use the Percent Only analysis type to estimate the number and estimated 
percentages of the disciplines taught by teacher educators. 

As with the previous examples, we first proceed to identify the variables relevant to 
the analysis in the appropriate files. We can look either into the codebook provided 
along with the data or in the educator questionnaire. Variable ITEDUGRP contains the 
information about the disciplines taught by educators. 

The merged data file DEG-TEDS.SAV containing only educator data will be used for 
this example. The variable that identifies the country (IDGNTRY), plus the variable 
of interest (ITEDUGRP) will be used as Grouping Variables in the lEA IDB Analyzer. 
The Weight Variable containing the educator sampling weights (FINWGTE) and the 
corresponding replicate weights (EINRWEl to FINRWE32) are selected automatically 
by the software. 

Figure 4.14 Example Educator Level Analysis Taken from the TEDS-M International 
Report (Exhibit 4.6) 


Exhibit 4.6: Disciplines taught by teacher educators (estimated percent) 


Country 

n 

A. Mathematics and 

B. General Pedagogy 

Both Areas 



Mathematics Pedagogy 



A. 

and B. 



Est . % 

(SE) 

Est . % 

(SE) 

Est . % 

(SE) 

Botswana 

43 

36.4 

( 0 . 0 ) 

63.6 

( 0 . 0 ) 

0.0 

( 0 . 0 ) 

Chile“ 

392 

18.0 

( 0 . 3 ) 

58.8 

( 0 . 6 ) 

23.1 

( 0 . 7 ) 

Chinese Taipei 

195 

40.4 

( 4 . 1 ) 

59.0 

( 4 . 1 ) 

0.6 

( 0 . 2 ) 

Georgia 

62 

65.6 

( 1 . 8 ) 

31.3 

( 0 . 3 ) 

3.1 

( 2 . 2 ) 

permany*’ 

482 

12.1 

( 3 . 2 ) 

62.0 

( 5 . 9 ) 

25.9 

( 4 . 6 ) 

^aiaysia' 

255 

59.1 

( 0 . 1 ) 

13.4 

( 0 . 0 ) 

27.5 

( 0 . 1 ) 

Oman'' 

84 

62.1 

( 0 . 1 ) 

35.9 

( 0 . 1 ) 

1.9 

( 0 . 0 ) 

Phiiippines 

589 

29.5 

( 3 . 0 ) 

46.0 

( 5 . 9 ) 

24.5 

( 5 . 4 ) 

Poiand' 

734 

64.9 

( 0 . 3 ) 

32.7 

( 0 . 2 ) 

2.4 

( 0 . 1 ) 

Russian Federation' 

1212 

76.7 

( 2 . 4 ) 

20.6 

( 1 . 9 ) 

2.7 

( 0 . 9 ) 

Singapore 

77 

33.0 

( 0 . 0 ) 

67.0 

( 0 . 0 ) 

0.0 

( 0 . 0 ) 

Spain^ 

533 

20.8 

( 0 . 7 ) 

76.1 

( 2 . 2 ) 

3.1 

( 2 . 4 ) 

^witzeriand'’ 

220 

18.5 

( 0 . 5 ) 

81.3 

( 0 . 4 ) 

0.2 

( 0 . 2 ) 

Thaiiand 

312 

39.0 

( 0 . 1 ) 

36.3 

( 0 . 1 ) 

24.8 

( 0 . 1 ) 


Notes: 

When reading this table, keep in mind the limitations listed earlier in this chapter and denoted in the table above by footnote 
letters. 

The shaded areas identify data that, for reasons explained in the list of limitations, cannot be compared with confidence to data 
from other countries. 
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To perform this analysis, proceed as follows (leading to the lEA IDE Analyzer 

configuration as in Figure 4.15): 

1 ) Open the analysis module of the lEA IDE Analyzer. 

2) Specify the data file DEG-TEDS.SAV as the Analysis File. 

3) Select Percentages only as the Analysis Type. 

4) The variable IDCNTRY is pre-selected as Grouping Variable by default. Add the 
variable ITEDUGRP as a second Grouping Variable. 

5) The Weight Variable is automatically identified by the software. As this example 
analysis uses educator data, FINWGTE is selected by default. The Replicate Weights 
variables FINRWEl to F1NRWE32 also are selected by default. 

6) Specify the name and folder of the output files in the Output Files field. 

7) Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Rim menu of SPSS 
and selecting the All menu option. If the syntax file already exisfs, the lEA IDE 
Analyzer will prompt you to confirm overwriting it. 


Figure 4.15 lEA IDB Analyzer Setup for the Example Educator Level Analysis 
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The results of this analysis are presented in Figure 4.16. The results by country comprise 
up to three lines, depending on the existence of educators with responsibility for 
mathematics and mathematics pedagogy or general pedagogy in the programs selected 
for the study. The results are presented in the same manner as in the previous examples, 
with countries identified in the first column and the second column describing the 
categories of ITEDUGRP. 

Figure 4.16 shows that 65.6% of the educators in Georgia teach mathematics and 
mathematics pedagogy; 31.3% are general pedagogy educators; and 3.1% of the 
educators in Georgia have responsibilities to teach in both disciplines. The appropriate 
standard errors are presented in the last column of Figure 4. 16 under the label “Percent 
(s.e.)”. 


Figure 4.16 Output for Educator Level Analysis Example 


Percentages by 
‘COUNTRY ID* 

(IDCNTRY ITEDUGRP) 
‘EDUCATOR GROUP* 

N of 
Cases 

Sum of 
FINWGTE 

Percent 

Percent 

(s.e.) 

Botswana 

MATH AND MATH PEDAGOGY EDUCATOR 

16 

16 

36.4 

. 0 


PEDAGOGY EDUCATOR 

27 

28 

63 . 6 

. 0 

Chile 

MATH AND MATH PEDAGOGY EDUCATOR 

83 

131 

18.0 

. 3 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

57 

169 

23 . 1 

. 7 


PEDAGOGY EDUCATOR 

252 

429 

58.8 

. 6 

Chinese Taipei 

MATH AND MATH PEDAGOGY EDUCATOR 

85 

137 

40.4 

4.1 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

2 

2 

. 6 

. 2 


PEDAGOGY EDUCATOR 

108 

200 

59.0 

4.1 

Georgia 

MATH AND MATH PEDAGOGY EDUCATOR 

41 

42 

65 . 6 

1 . 8 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

1 

2 

3 . 1 

2.2 


PEDAGOGY EDUCATOR 

20 

20 

31.3 

. 3 

Germany 

MATH AND MATH PEDAGOGY EDUCATOR 

115 

476 

12 . 1 

3 . 2 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

142 

1022 

25 . 9 

4.6 


PEDAGOGY EDUCATOR 

225 

2445 

62.0 

5 . 9 

Malaysia 

MATH AND MATH PEDAGOGY EDUCATOR 

165 

270 

59.1 

. 1 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

69 

125 

27.5 

. 1 


PEDAGOGY EDUCATOR 

21 

61 

13.4 

. 0 

Oman 

MATH AND MATH PEDAGOGY EDUCATOR 

51 

64 

62 . 1 

. 1 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

2 

2 

1 . 9 

. 0 


PEDAGOGY EDUCATOR 

31 

37 

35 . 9 

. 1 

Philippines 

MATH AND MATH PEDAGOGY EDUCATOR 

194 

840 

29.5 

3 . 0 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

116 

698 

24.5 

5.4 


PEDAGOGY EDUCATOR 

279 

1309 

46.0 

5 . 9 

Poland 

MATH AND MATH PEDAGOGY EDUCATOR 

455 

766 

64.9 

. 3 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

24 

28 

2.4 

. 1 


PEDAGOGY EDUCATOR 

255 

386 

32 . 7 

. 2 

Russian 

MATH AND MATH PEDAGOGY EDUCATOR 

920 

2405 

76.7 

2.4 

Federation 

EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

17 

85 

2 . 7 

. 9 


PEDAGOGY EDUCATOR 

275 

646 

20.6 

1 . 9 

Singapore 

MATH AND MATH PEDAGOGY EDUCATOR 

25 

30 

33.0 

. 0 


PEDAGOGY EDUCATOR 

52 

61 

67.0 

. 0 

Spain 

MATH AND MATH PEDAGOGY EDUCATOR 

120 

160 

20.8 

. 7 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

13 

24 

3 . 1 

2.4 


PEDAGOGY EDUCATOR 

400 

586 

76 . 1 

2.2 

Switzerland 

MATH AND MATH PEDAGOGY EDUCATOR 

51 

77 

18.5 

.5 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

1 

1 

. 2 

. 2 


PEDAGOGY EDUCATOR 

168 

338 

81.3 

.4 

Thailand 

MATH AND MATH PEDAGOGY EDUCATOR 

121 

138 

39.0 

. 1 


EDUCATOR WHO HAS BOTH RESPONSIBILITIES 

74 

88 

24.8 

. 1 


PEDAGOGY EDUCATOR 

117 

128 

36.3 

. 1 

X . International 

MATH AND MATH PEDAGOGY EDUCATOR 



36.0 

.4 

Average 

EDUCATOR WHO HAS BOTH RESPONSIBILITIES 



8.7 

.5 


PEDAGOGY EDUCATOR 



42 . 8 

. 6 
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4.7. TEDS-M Analysis with Institution or Program Level Data 

Analysis of the institutional-level data can be used, for example, to make statements 
about the number or percentages of future teachers attending specific programs in 
certain institutions with certain characteristics. In this sense, the characteristics of the 
specific programs at the institution become an attribute of the future teachers. Our 
example of institution or program level analysis will investigate the mathematics 
pedagogy content knowledge of future teachers with regards to the importance given 
to their performance on the admission examination that they took when applying to 
the institution. The knowledge scores are included in the future primary teacher data 
file and the information about the importance of the admission examination results is 
included in the institutional program data file. 

To conduct this analysis, we will use the Percentages and Means analysis type available 
from the analysis module of the lEA IDE Analyzer. The first step in our analysis is to 
identify the variables of interest from the questionnaires and review the documentation 
about specific national adaptations to the questions of interest (see Supplements 1 and 
2 of this IDE User Guide). The variable MIC002C in the institutional program data file 
contains information on how important the future teachers’ admission examination 
result is judged to be by the institution for their selection in the respective teacher 
preparation program. The mathematics pedagogy content knowledge from the future 
primary teacher data file will be used as the analysis variable. This way we will calculate 
the mean mathematics pedagogy content knowledge of future teachers studying in 
programs for which admission examinations results were “not considered”, or were 
considered “not very important”, “somewhat important”, and “very important” for the 
selection of the candidates. 

The data to be used comprise the combined future primary teacher and institutional 
program data for all countries (DIG&DPG-TEDS.SAV). The variable that identifies the 
country (IDGNTRY) is selected automatically by the lEA IDE Analyzer, as well as the 
weight variables that contain the sampling information used to generate the replicate 
weights for the analysis. 

To perform this analysis, proceed as follows (see the lEA IDE Analyzer configuration 
in Figure 4.17): 

1 ) Open the analysis module of the lEA IDE Analyzer. 

2) Specify the data file DIG&DPG-TEDS.SAV as the Analysis File. 

3) Select Percentages and Means as the Analysis Type. 

4) Make sure the Exclude Missing From Analysis check box is marked. 

5) Add the variable TARGET? as a second Grouping Variable (in addition to 
IDGNTRY). Furthermore, add variable MIG002G as another Grouping Variable 

6) Glick on the Analysis Variables and select the variable MPCK from the list of 
available variables and move it to the Analysis Variables field by clicking the right 
arrow button in this section. 

7) The Weight Variable is automatically defined by the software, because this 
example analysis uses combined institutional program and future primary teacher 
data with institutional program data disaggregated to the future teacher level, 
the future primary teacher data weight (FINWGTP) is selected by default. The 
associated future primary teacher Replicate Weights (FINRWPl to FINRWP32) 
also are selected by default. 
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8) Specify the name and folder of the output files in the Output Files field. 

9) Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS 
syntax window. The syntax file will be executed by opening the Run menu of SPSS 
and selecting the All menu option. If the syntax file already exists, the lEA IDB 
Analyzer will prompt you to confirm overwriting the file. 


Figure 4.17 lEA IDB Analyzer Setup for Example of Institutional Program Level Analysis 



The results of this analysis are presented in Figure 4.18. In this example, each country’s 
results are separated by program group and presented on up to four lines, one for each 
value of the M1C002C variable that occurs in the data. The results are presented in 
the same manner as in the previous examples, with countries identified in the first 
column, program groups in the second, and the categories of M1C002C in the third 
column. Figure 4.18 shows that in the majority of countries future primary teachers 
are in general not likely to have higher mathematics pedagogy content knowledge with 
increasing importance given to admission examinations in the institutions. In Georgia 
the results of entrance examinations were considered very important by 100% of the 
institutions, whereas in Singapore and Norway (AFU) all future primary teachers were 
from institutions that did not consider admission examination results at all. In Chile, 
Chinese Taipei, the Philippines, and Norway (AFU) the mathematics pedagogy content 
knowledge of future primary teachers is lower, with increased importance placed 
on the results of admission examinations. Botswana, Poland, Russian Federation, 
Thailand, and the United States show no consistent pattern. In Spain mathematics 
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pedagogy content knowledge is almost the same regardless of the importance placed 
on examination results. In Malaysia and Switzerland mathematics pedagogy content 
knowledge scores tend to increase as the importance placed on admission examinations 
increases. However, the standard errors have to he taken into account. It should be 
noted that this type of analysis does not show whether the differences between groups 
are statistically significant. 


Figure 4.18 Output for Example Institutional Program Level Analysis 
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